CN117687011A - Indoor personnel detection method and device, electronic equipment and storage medium - Google Patents

Indoor personnel detection method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN117687011A
CN117687011A CN202311494857.9A CN202311494857A CN117687011A CN 117687011 A CN117687011 A CN 117687011A CN 202311494857 A CN202311494857 A CN 202311494857A CN 117687011 A CN117687011 A CN 117687011A
Authority
CN
China
Prior art keywords
information
indoor
millimeter wave
wave radar
tracking
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311494857.9A
Other languages
Chinese (zh)
Inventor
刘子明
罗雨泉
袁常顺
王俊
麦超云
郑瑶
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Innovation Research Institute of Beihang University
Original Assignee
Hangzhou Innovation Research Institute of Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Innovation Research Institute of Beihang University filed Critical Hangzhou Innovation Research Institute of Beihang University
Priority to CN202311494857.9A priority Critical patent/CN117687011A/en
Publication of CN117687011A publication Critical patent/CN117687011A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/06—Systems determining position data of a target
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/50—Systems of measurement based on relative movement of target
    • G01S13/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/50—Systems of measurement based on relative movement of target
    • G01S13/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
    • G01S13/62—Sense-of-movement determination
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G01S7/41—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

The embodiment of the invention provides an indoor personnel detection method and device, electronic equipment and a storage medium. The method comprises the following steps: sampling and quantizing echo signals of personnel in the millimeter wave radar detection room through an analog-to-digital converter to obtain time domain signals; carrying out digital signal processing on the time domain signals, and generating point cloud data with space coordinate information by combining a maximum value searching algorithm and a minimum selection constant false alarm SO-CFAR algorithm; clustering the point cloud data by adopting a DBSCAN clustering algorithm to obtain a clustering result; filtering interference information from the clustering result, wherein the interference information is obtained by scanning and determining an interfering object in an indoor environment by a millimeter wave radar; the clustering result of filtering the interference information is tracked by adopting Kalman filtering to obtain the tracking track of the indoor personnel, and based on the tracking track, the embodiment of the invention can accurately position the position and the state of the indoor personnel under the condition of protecting privacy.

Description

Indoor personnel detection method and device, electronic equipment and storage medium
Technical Field
The embodiment of the invention relates to the technical field of millimeter wave radar positioning, in particular to an indoor personnel detection method and device, electronic equipment and a storage medium.
Background
Currently, vision-based localization techniques are used to detect and localize targets by image processing and computer vision techniques. However, the implementation of the positioning technology based on vision requires deployment of a camera, and the camera can directly acquire sensitive information such as human body images or videos, so that the problem of privacy disclosure exists, and therefore the positioning technology is not applicable to a scene requiring privacy protection. In addition, the vision-based positioning technology also has certain requirements on environmental conditions such as temperature, humidity, illumination and the like, and the performance is unstable under external interference, so that the accuracy of positioning the position and state of indoor personnel is affected. Therefore, how to accurately locate the position and state of indoor personnel under the condition of protecting privacy is a technical problem to be solved.
Disclosure of Invention
The embodiment of the invention provides an indoor personnel detection method and device, electronic equipment and a storage medium, which can accurately position the position and state of indoor personnel under the condition of protecting privacy.
In a first aspect, an embodiment of the present invention provides an indoor person detection method, including:
sampling and quantizing echo signals of personnel in the millimeter wave radar detection room through an analog-to-digital converter to obtain time domain signals;
Carrying out digital signal processing on the time domain signal, and generating point cloud data with space coordinate information by combining a maximum value searching algorithm and a minimum selection constant false alarm SO-CFAR algorithm;
clustering the point cloud data by adopting a DBSCAN clustering algorithm to obtain a clustering result;
filtering interference information from the clustering result, wherein the interference information is obtained by scanning and determining an interfering object in an indoor environment by the millimeter wave radar;
and tracking the clustering result of which the interference information is filtered by adopting Kalman filtering to obtain the tracking track of the indoor personnel.
In some embodiments, the method further comprises:
determining a judging area used for judging whether indoor personnel enter or leave, a first loiter area located indoors and a second loiter area located outdoors in an indoor environment, wherein the judging area is located between a first mixing line and a second mixing line, the first mixing line is located between the first loiter area and the judging area, and the second mixing line is located between the judging area and the second loiter area;
under the condition that the tracking track of the indoor personnel sequentially passes through the first mixing line and the second mixing line, the number of people in the indoor personnel is counted and subtracted by one;
And under the condition that the tracking track of the indoor personnel sequentially passes through the second mixing line and the first mixing line, adding one to the statistics of the number of people in the indoor personnel.
In some embodiments, the point cloud data includes straight line distance information, speed information, energy information, azimuth angle information and pitch angle information, the digital signal processing is performed on the time domain signal, and the point cloud data with space coordinate information is generated by combining a maximum value search algorithm and a minimum selection constant false alarm SO-CFAR algorithm, including:
performing distance Fourier transform and Doppler Fourier transform on the time domain signal sequentially to generate a distance Doppler two-dimensional matrix, wherein the distance Doppler two-dimensional matrix comprises a distance dimension and a Doppler dimension;
determining linear distance information from a target point to a millimeter wave radar in the indoor environment according to the index value of the distance dimension;
determining the speed information of the target point according to the index value of the Doppler dimension;
obtaining energy information of the target point by taking a model of the distance Doppler two-dimensional matrix result;
and searching the number of the target points and corresponding subscripts by combining a maximum value searching algorithm and a minimum selection constant false alarm SO-CFAR algorithm, performing Fourier transformation according to the antenna dimension of the millimeter wave radar, and obtaining azimuth angle information and pitch angle information of the target points through coordinate transformation.
In some embodiments, the method further comprises:
the millimeter wave radar scans the indoor environment for a plurality of times to obtain newly added interference point cloud data;
storing the newly added interference point cloud data to a new cluster, and comparing the newly added interference point cloud data with interference information pre-stored in a Flash memory of the millimeter wave radar to obtain a comparison result;
under the condition that the comparison result is that the newly added interference point cloud data is similar to the interference information, the newly added interference point cloud data is stored in the Flash memory;
and under the condition that the comparison result is that the newly added interference point cloud data is dissimilar to the interference information, continuing to scan the indoor environment for the next time until the scanning frequency threshold is reached.
In some embodiments, the tracking the clustering result with the interference information filtered by using kalman filtering to obtain a tracking track of the indoor personnel includes:
estimating the state of the tracking group centroid at time n based on the state of time n-1 and a process noise covariance matrix by using a prediction process of Kalman filtering;
in the event that it is determined that there are one or more tracks and associated predicted state vectors, calculating a normalized distance function as a cost function for observations within a Gate of predicted centroids to associate an observation with each track, and minimizing the cost function during the allocation process, each time allocating an observation to the nearest track, allocating a series of point clouds for each tracked track;
And when the measured value at the moment n is obtained, updating the prediction state vector and the prediction error covariance estimation through Kalman filtering to obtain the tracking track of the indoor personnel.
In some embodiments, the method further comprises:
for observations outside any existing Gate, a new group tracker will be assigned and initialized;
selecting a forefront observation value, setting a centroid equal to the forefront observation value, screening other candidate observation values through the radial speed of the forefront observation value, recalculating the centroid under the condition that the candidate observation values are checked to be in a speed range, and adding the candidate observation values into the tracking group;
if it is determined that the number of observations in the tracking group meets a preset number threshold, the combined signal-to-noise ratio SNR is higher than the preset SNR threshold, and the dynamic measure of the centroid meets a preset measure threshold, a new tracking object is allocated and the discrete matrix is initialized using the associated observations in the tracking group.
In some embodiments, the millimeter wave radar is a 60GHz frequency modulated continuous wave millimeter wave radar with a range resolution of 5cm and a speed resolution of 0.1m/s, the millimeter wave radar using a 4x4 MIMO array antenna.
In a second aspect, an embodiment of the present invention further provides an indoor personnel detection apparatus, where the apparatus includes:
the sampling module is used for sampling and quantizing echo signals of personnel in the millimeter wave radar detection room through the analog-to-digital converter to obtain time domain signals;
the generation module is used for carrying out digital signal processing on the time domain signal, and generating point cloud data with space coordinate information by combining a maximum value search algorithm and a minimum selection constant false alarm SO-CFAR algorithm;
the clustering module is used for clustering the point cloud data by adopting a DBSCAN clustering algorithm to obtain a clustering result;
the filtering module is used for filtering interference information from the clustering result, wherein the interference information is obtained by scanning and determining an interfering object in an indoor environment by the millimeter wave radar;
and the tracking module is used for tracking the clustering result with the interference information filtered by adopting Kalman filtering to obtain the tracking track of the indoor personnel.
In a third aspect, an embodiment of the present invention further provides an electronic device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the indoor person detection method according to the first aspect when executing the computer program.
In a fourth aspect, embodiments of the present invention further provide a computer-readable storage medium storing computer-executable instructions for performing the indoor person detection method according to the first aspect.
According to the indoor personnel detection method and device, the electronic equipment and the storage medium provided by the embodiment of the invention, the indoor personnel detection method comprises the following steps: sampling and quantizing echo signals of personnel in the millimeter wave radar detection room through an analog-to-digital converter to obtain time domain signals; carrying out digital signal processing on the time domain signals, and generating point cloud data with space coordinate information by combining a maximum value searching algorithm and a minimum selection constant false alarm SO-CFAR algorithm; clustering the point cloud data by adopting a DBSCAN clustering algorithm to obtain a clustering result; filtering interference information from the clustering result, wherein the interference information is obtained by scanning and determining an interfering object in an indoor environment by a millimeter wave radar; and tracking the clustering result of the filtered interference information by adopting Kalman filtering to obtain the tracking track of the indoor personnel. Based on the above, the embodiment of the invention is suitable for a scene requiring privacy protection, after the radar antenna receives the echo signal of the millimeter wave radar detection indoor personnel, the echo signal is sampled and quantized through the analog-to-digital converter to obtain the time domain signal, and the time domain signal is processed through the digital signal, because the traditional CFAR algorithm can not cause the existence of the human body to be detected in the sleep mode, the processing of the time domain signal is combined with the minimum selection constant false alarm SO-CFAR algorithm to generate the point cloud data with space coordinate information, thereby accurately detecting the position, the speed, the direction and other information of the target, scanning and filtering the interference information of the flower, the grass and the curtain and the like through the indoor environment, and judging the position of the indoor personnel in the time dimension through the Kalman filtering SO as to achieve more effective human body positioning. Based on the method, the device and the system, the position and the state of indoor personnel can be accurately positioned under the condition of privacy protection.
Drawings
FIG. 1 is a flow chart of a method for detecting indoor personnel according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a millimeter wave radar indoor detection system provided by an embodiment of the present invention;
FIG. 3 is a flow chart of a method for detecting indoor personnel according to another embodiment of the present invention;
fig. 4A is a top view of a millimeter wave radar installation provided by one embodiment of the present invention;
fig. 4B is a schematic diagram of a millimeter wave radar detection area and a hybrid line provided in one embodiment of the present invention;
FIG. 5 is a flow chart of step S102 of FIG. 1 provided in one embodiment of the present invention;
FIG. 6 is a flow chart of millimeter wave radar signal processing and data processing provided by one embodiment of the present invention;
FIG. 7 is a flow chart of a method for detecting indoor personnel according to another embodiment of the present invention;
FIG. 8 is a flow chart of learning indoor environment interference locations by millimeter wave radar provided by one embodiment of the present invention;
FIG. 9 is a flow chart of step S105 of FIG. 1 provided by one embodiment of the present invention;
FIG. 10 is a flowchart of a group target tracking algorithm provided by one embodiment of the present invention;
FIG. 11 is a flow chart of a method for detecting indoor personnel according to another embodiment of the present invention;
FIG. 12 is a schematic view of an indoor personnel detection apparatus provided in one embodiment of the present invention;
Fig. 13 is a schematic diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
It should be noted that although functional block division is performed in a device diagram and a logic sequence is shown in a flowchart, in some cases, the steps shown or described may be performed in a different order than the block division in the device, or in the flowchart. The terms first, second and the like in the description and in the claims and in the following figures, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
In embodiments of the invention, the words "further," "exemplary," or "optionally," etc. are used to indicate by way of example, illustration, or description, and should not be construed as preferred or advantageous over other embodiments or designs. The use of the words "further," "exemplary," or "optionally" and the like is intended to present the relevant concepts in a concrete fashion.
First, several terms involved in the present invention are parsed:
FMCW: (Frequency Modulated Continuous Wave) frequency modulated continuous wave;
DBSCAN: (Density-Based Spatial Clustering of Applications with Noise) Density-based clustering algorithm;
ADC: (analog-to-digital converter) analog-to-digital converter;
SPI: (Serial Peripheral interface) a serial peripheral interface;
FFT: (Fast Fourier Transform) fast fourier transform.
In order to more conveniently describe the working principle of the embodiment of the present invention, an introduction of the related technical scenario is given below.
Indoor millimeter wave radar is an advanced radar technology, and has been widely focused in recent years due to its unique performance and application scenario. Millimeter wave radar refers to radar with a working frequency between 30 and 300GHz, while indoor millimeter wave radar is millimeter wave radar used in indoor environments.
Compared with the traditional radar, the indoor millimeter wave radar has higher frequency band, smaller wavelength and higher resolution, thus having stronger anti-interference capability and more accurate ranging precision. In addition, the indoor millimeter wave radar has the advantages of small volume, light weight, low power consumption and the like, so that the indoor millimeter wave radar is more suitable for portable and embedded applications.
Smart home is one of the important application fields of indoor millimeter wave radar. Compared with a camera, the millimeter wave radar has the advantages that the problem of privacy disclosure is not related to, the design of an uncomfortable lens is not provided for the appearance of the millimeter wave radar, more importantly, the data information of the radar is completely anonymous, the related privacy regulation requirements are met, and the millimeter wave radar can be applied to more office scenes which are not suitable for deploying the camera. Through embedding millimeter wave radar into intelligent home system, can realize automatic control and monitoring, improve comfort level and the convenience of house life. For example, in an intelligent lighting system, millimeter wave radar can detect the position and movement of a human body, and the functions of automatically switching on and off a lamp, adjusting brightness and the like are realized.
In the related art, the vision-based localization technique is to implement object detection and localization through image processing and computer vision techniques. However, the implementation of the positioning technology based on vision requires deployment of a camera, and the camera can directly acquire sensitive information such as human body images or videos, so that the problem of privacy disclosure exists, and therefore the positioning technology is not applicable to a scene requiring privacy protection. In addition, the vision-based positioning technology also has certain requirements on environmental conditions such as temperature, humidity, illumination and the like, and the performance is unstable under external interference, so that the accuracy of positioning the position and state of indoor personnel is affected. Therefore, how to accurately locate the position and state of indoor personnel under the condition of protecting privacy is a technical problem to be solved.
Based on the detection method and device, electronic equipment and storage medium are provided. The indoor personnel detection method comprises the following steps: sampling and quantizing echo signals of personnel in the millimeter wave radar detection room through an analog-to-digital converter to obtain time domain signals; carrying out digital signal processing on the time domain signals, and generating point cloud data with space coordinate information by combining a maximum value searching algorithm and a minimum selection constant false alarm SO-CFAR algorithm; clustering the point cloud data by adopting a DBSCAN clustering algorithm to obtain a clustering result; filtering interference information from the clustering result, wherein the interference information is obtained by scanning and determining an interfering object in an indoor environment by a millimeter wave radar; and tracking the clustering result of the filtered interference information by adopting Kalman filtering to obtain the tracking track of the indoor personnel. Based on the above, the embodiment of the invention is suitable for a scene requiring privacy protection, after the radar antenna receives the echo signal of the millimeter wave radar detection indoor personnel, the echo signal is sampled and quantized through the analog-to-digital converter to obtain the time domain signal, and the time domain signal is processed through the digital signal, because the traditional CFAR algorithm can not cause the existence of the human body to be detected in the sleep mode, the processing of the time domain signal is combined with the minimum selection constant false alarm SO-CFAR algorithm to generate the point cloud data with space coordinate information, thereby accurately detecting the position, the speed, the direction and other information of the target, scanning and filtering the interference information of the flower, the grass and the curtain and the like through the indoor environment, and judging the position of the indoor personnel in the time dimension through the Kalman filtering SO as to achieve more effective human body positioning. Based on the method, the device and the system, the position and the state of indoor personnel can be accurately positioned under the condition of privacy protection.
Embodiments of the present invention will be further described below with reference to the accompanying drawings.
As shown in fig. 1, fig. 1 is a flowchart of an indoor person detection method according to an embodiment of the present invention, which may include, but is not limited to, steps S101 to S105.
Step S101: sampling and quantizing echo signals of personnel in the millimeter wave radar detection room through an analog-to-digital converter to obtain time domain signals;
step S102: carrying out digital signal processing on the time domain signals, and generating point cloud data with space coordinate information by combining a maximum value searching algorithm and a minimum selection constant false alarm SO-CFAR algorithm;
step S103: clustering the point cloud data by adopting a DBSCAN clustering algorithm to obtain a clustering result;
step S104: filtering interference information from the clustering result, wherein the interference information is obtained by scanning and determining an interfering object in an indoor environment by a millimeter wave radar;
step S105: and tracking the clustering result of the filtered interference information by adopting Kalman filtering to obtain the tracking track of the indoor personnel.
In one embodiment, the millimeter wave radar is a 60GHz frequency modulated continuous wave millimeter wave radar with a range resolution of 5cm and a speed resolution of 0.1m/s, and the millimeter wave radar uses 4x4 MIMO array antennas, 4 transmit and 4 receive channels and 16 channels. The millimeter wave radar chip and the singlechip are communicated through SPI to set a register in the radar chip, an echo signal of the millimeter wave radar is sampled and quantized through an ADC of the singlechip, and then digital signal processing is carried out to generate point cloud data with space coordinate information.
In an embodiment, the invention adopts the domestic 60G millimeter wave radar four-transmitting and four-receiving radio frequency chip, can accurately identify the person, and can accurately position the position and state of the person indoors. The method comprises the steps of performing signal processing on waveforms received by a radar antenna to preliminarily obtain the approximate conditions of an indoor environment and the positions of indoor human bodies, and judging the positions of the human bodies in a time dimension through Kalman filtering and a group target tracking algorithm so as to achieve more effective human body positioning.
In an embodiment, as shown in fig. 2, in the smart home scenario, the radar module communicates with the bluetooth module through a serial port, and then the bluetooth module transmits the received data to the server. Firstly, the radar module detects the movement or position change of the human body, and the detected data is sent to the Bluetooth module through the serial port. And secondly, after the Bluetooth module receives the data from the radar module, the data is transmitted to the server in a penetrating way (namely, the data is transmitted without processing). And then the server processes and analyzes the transmitted data after receiving the transmitted data, for example, decodes and recognizes the data so as to realize corresponding functions, such as human body induction, intelligent control and the like. And finally, the server performs corresponding operations according to the analyzed and processed data, such as controlling the working state of the intelligent household equipment, or storing the data into a database for subsequent analysis and calling. And after receiving the data, the server sends the operation result or feedback information to the Bluetooth module through the serial port. And after receiving the feedback information from the server, the Bluetooth module transmits the feedback information to the radar module. After receiving the feedback information, the radar module performs corresponding processing, such as adjusting the detection range, updating the target position, and the like, as required. In the whole process, the radar module is responsible for detecting the movement and position change of a human body and the number of people in a range, and sending the detected data to the Bluetooth module; the Bluetooth module is responsible for transmitting the received data to the server; and the server processes and analyzes the received data and performs corresponding operation or feedback control according to the result.
It should be noted that, the embodiment of the invention is suitable for a scene requiring privacy protection, after the radar antenna receives the echo signal of the millimeter wave radar detection indoor personnel, the echo signal is sampled and quantized through the analog-to-digital converter to obtain a time domain signal, and the time domain signal is processed through digital signal processing, because the traditional CFAR algorithm can not cause the existence of the human body to be detected in the sleep mode, the processing of the time domain signal is combined with the minimum selection constant false alarm SO-CFAR algorithm to generate the point cloud data with space coordinate information, SO that the information of the position, the speed, the direction and the like of the target can be accurately detected, the interference information such as a flower curtain and the like is filtered through scanning the indoor environment, and the position of the indoor personnel is judged in the time dimension through Kalman filtering, SO that more effective human body positioning is achieved. Based on the method, the device and the system, the position and the state of indoor personnel can be accurately positioned under the condition of privacy protection.
In the case of further considering privacy, if no positioning information is needed, the number of people in the room can be obtained by counting the entrance and exit, as shown in fig. 3, the indoor people detection method of the present invention may further include, but is not limited to, steps S301 to S303.
Step S301: determining a judging area used for judging whether indoor personnel enter or leave in an indoor environment, a first loiter area located indoors and a second loiter area located outdoors, wherein the judging area is located between a first mixing line and a second mixing line, the first mixing line is located between the first loiter area and the judging area, and the second mixing line is located between the judging area and the second loiter area;
step S302: under the condition that the tracking track of indoor personnel is determined to sequentially pass through the first mixing line and the second mixing line, the number of people in the indoor personnel is counted and subtracted by one;
step S303: and under the condition that the tracking track of the indoor personnel is determined to sequentially pass through the second mixing line and the first mixing line, adding one to the statistics of the number of the indoor personnel.
In an embodiment, if the user privacy awareness is strong, and the track generated by the millimeter wave radar is considered to violate the personal privacy, only the total indoor people are displayed through track judgment, and then the indoor people are required to be counted through a mixed line detection method. For convenience of installation and simplicity of maintenance, the millimeter wave radar may be installed on a ceiling, and fig. 4A shows a top view of the installation, with the radar in the middle, and the circle indicating the maximum range that the radar can detect. Fig. 4B is a schematic diagram after setting of a mixing line, wherein a dotted rectangle box indicates a door, a mixing line B is a first mixing line, a mixing line c is a second mixing line, a first loiter area located indoors is located between a mixing line a and a mixing line B, and a second loiter area located outdoors is located between a mixing line c and a mixing line d.
The judging area is judged to be positioned between the mixing lines bc, and people can go outwards only when people pass through the mixing line b and then pass through the mixing line c, namely the number of indoor people is reduced by one; the people walk inwards through the mixing line c and then through the mixing line b, namely the number of people in the room is increased by one.
It should be noted that, since a person may wander at a doorway and does not want to enter a room, however, a person walking back and forth by radar detection may cause other targets to be more than one, so that radar counts are disordered, and thus a wandering area is added. If the multi-target track is found to be continuously walking left and right but does not pass through the mixed line a and the mixed line d in the loitering area, the track is defined as loitering personnel. Loiter personnel do not count when passing through the judgment area.
As shown in fig. 5, step S102 is further described, and step S102 includes, but is not limited to, steps S501 to S505.
Step S501: sequentially performing distance Fourier transform and Doppler Fourier transform on the time domain signals to generate a distance Doppler two-dimensional matrix, wherein the distance Doppler two-dimensional matrix comprises a distance dimension and a Doppler dimension;
step S502: determining linear distance information from a target point to the millimeter wave radar in the indoor environment according to the index value of the distance dimension;
Step S503: determining the speed information of the target point according to the index value of the Doppler dimension;
step S504: obtaining energy information of a target point by taking a model of a distance Doppler two-dimensional matrix result;
step S505: and searching the number of target points and corresponding subscripts by combining a maximum value searching algorithm and a minimum selection constant false alarm SO-CFAR algorithm, performing Fourier transformation according to the antenna dimension of the millimeter wave radar, and obtaining azimuth angle information and pitch angle information of the target points through coordinate transformation.
In one embodiment, as shown in fig. 6, the time domain signal is acquired by the ADC and then subjected to one-dimensional FFT and two-dimensional FFT to obtain the distance velocity information. The specific signal processing flow is to firstly select the channel with the best signal-to-noise ratio of 16 channels as a detection channel, sequentially perform a distance-to-FFT and a Doppler-to-FFT on the time domain signals acquired by the ADC to generate a distance-to-Doppler two-dimensional matrix, obtain the linear distance information R from the target point to the radar through the index value on the distance dimension, obtain the speed information VEL of the target point through the index value on the Doppler dimension, and obtain the energy information POW of the target point by taking the modulus of the two-dimensional FFT result. After the two-dimensional FFT data are obtained, the existence of a human body cannot be detected in a sleep mode by a traditional CFAR, SO that a method of combining a maximum value searching method with an SO-CFAR is added to find out the number of target points and corresponding subscripts, and the azimuth information Azim and Elev of the target points can be obtained by performing FFT according to the antenna dimension. The signal processing part finishes obtaining the point cloud data generated by the millimeter wave radar.
As shown in fig. 7, the indoor person detection method of the present invention may further include, but is not limited to, steps S701 to S704.
Step S701: the indoor environment is scanned for multiple times through the millimeter wave radar, and newly added interference point cloud data are obtained;
step S702: storing the newly-added interference point cloud data to a new cluster, and comparing the newly-added interference point cloud data with interference information pre-stored in a Flash memory of the millimeter wave radar to obtain a comparison result;
step S703: under the condition that the comparison result is that the newly added interference point cloud data is similar to the interference information, the newly added interference point cloud data is stored in a Flash memory;
step S704: and under the condition that the comparison result is that the newly added interference point cloud data is dissimilar to the interference information, continuing to scan the indoor environment for the next time until the scanning frequency threshold is reached.
In an embodiment, as shown in fig. 8, by configuring the radar to enter the environment adaptation mode, other interference objects such as flowers and plants can be filtered through the following diagram, so as to avoid false alarms of the radar. Firstly, a command for starting learning is sent to the radar, the millimeter wave radar scans the surrounding environment, if a micro-motion object exists, the radar is marked as flowers and plants or a curtain, but the detection of an interfering object or a micro-motion human body cannot be ensured, so that the marked interfering object needs to be judged for many times for a long time.
Firstly, the environment is scanned to obtain point clouds which are possibly interfered, then the point clouds are stored in a newCluster, if the interference information is stored before the flash, the point clouds are compared, and if the interference information is similar, the fact that the learning is correct and the latest interference data are stored in the flash is indicated. If the interference information is not stored before the flash or is inconsistent with the newCluster, scanning is carried out for the next time, scanning is carried out for five times, and the results of the five times are similar, and the finest newCluster is stored into the flash. Therefore, the method can avoid the situation that the human body is used as interference to be removed in the radar learning scanning process and accurately identify the interference position, and is convenient to screen out possible interference in the radar working operation process.
As shown in fig. 9, step S105 is further described, and step S105 includes, but is not limited to, steps S901 to S903.
S901: estimating the state of the tracking group centroid at time n based on the state of time n-1 and a process noise covariance matrix by using a prediction process of Kalman filtering;
s902: in the event that it is determined that there are one or more tracks and associated predicted state vectors, calculating a normalized distance function as a cost function for observations within a Gate of predicted centroids to associate an observation with each track, and minimizing the cost function during the allocation process, each time allocating an observation to the nearest track, allocating a series of point clouds for each tracked track;
S903: and when the measured value at the moment n is obtained, updating the prediction state vector and the prediction error covariance estimation through Kalman filtering to obtain the tracking track of the indoor personnel.
In one embodiment, as shown in FIG. 10, the prediction process using the Kalman filter estimates the state of the tracking group centroid at time n based on the state of time n-1 and the process noise covariance matrix. Each tracking target obtains an a priori state and prediction error covariance estimate. At the same time, the observation value vector corresponding to the prediction state is calculated in the process.
It is assumed that there are one or more tracks and associated predicted state vectors. For each given trajectory, a Gate (Gate) is formed with respect to the predicted centroid. Gate should comprehensively consider target mobility, degree of dispersion of groups, and measurement noise. And establishing an ellipsoid of the tracking group three-dimensional measurement centroid by using, namely updating the error covariance matrix. Ellipsoids will be used to represent a threshold function to evaluate the measured value we observe at time n. For observations within Gate, a normalized distance function is calculated as a cost function to correlate the observations with each trace. The assignment process will minimize the cost function, assigning one observation at a time to the nearest neighbor trajectory. Eventually, each tracking trajectory will be assigned a series of point clouds.
When the measurement at time n is obtained, the system state vector and the prediction error covariance estimate are updated by Kalman filtering.
As shown in fig. 11, the indoor person detection method of the present invention may further include, but is not limited to, steps S1101 to S1103.
S1101: for observations outside any existing Gate, a new group tracker will be assigned and initialized;
s1102: selecting a forefront observation value, setting a centroid equal to the forefront observation value, screening other candidate observation values through the radial speed of the forefront observation value, recalculating the centroid under the condition that the candidate observation values are checked to be in a speed range, and adding the candidate observation values into a tracking group;
s1103: in the case that the number of observations in the tracking group is determined to meet a preset number threshold, the combined signal-to-noise ratio SNR is higher than the preset SNR threshold, and the dynamic measure of the centroid meets a preset measure threshold, a new tracking object is assigned and the associated observations in the tracking group are used to initialize the discrete matrix.
In one embodiment, for observations that are not associated with any trace (outside of any of the existing threshold gates), a new group tracker will be assigned and initialized. This is an iterative process similar to the DBSCAN clustering algorithm. It is much simpler because it only does for the remaining observations. First a foremost observation is selected and a centroid equal to it is set. The radial velocity of this observation is used to further screen other candidate observations. Every time there is a candidate observation, we check if this point is within the speed range (i.e. speed check, performed after distance check). If the check passes, the centroid is recalculated and the point is added to the group. Once this step is completed, a few specific tests will be performed for the group. It is necessary to check whether the number of observations in the group meets the minimum requirement, whether the combined signal-to-noise ratio SNR is sufficiently high, and whether the measure of the dynamics of the centroid meets the minimum requirement, etc. If the tests are passed, a new tracking object will be assigned and the associated observations in the group will be used to initialize the discrete matrix. Groups with fewer associated observations will be ignored.
Based on the above, compared with other existing sensors, the sensor has at least the following advantages in terms of precision and resolution, environmental adaptability and privacy protection:
1. precision and resolution: millimeter wave radars have higher accuracy and resolution, and are able to accurately detect information such as the position, speed, and direction of a target, while other sensor schemes may not be as accurate and resolution as millimeter wave radars.
2. Environmental suitability: the millimeter wave radar can adapt to different environmental conditions, such as temperature, humidity, illumination and the like, and can resist external interference such as electromagnetic interference and the like, and other sensor schemes may have higher requirements on the environmental conditions or unstable performance under the influence of the external interference.
3. Privacy protection: millimeter wave radars do not involve privacy disclosure problems, while other sensor schemes may require sensitive information such as images or video to be acquired, with privacy protection issues.
In addition, as shown in fig. 12, an embodiment of the present invention further discloses an indoor personnel detection apparatus, which includes:
the sampling module 110 is configured to sample and quantize an echo signal of a person in the millimeter wave radar detection room through an analog-to-digital converter, so as to obtain a time domain signal;
The generating module 120 is configured to perform digital signal processing on the time domain signal, and generate point cloud data with spatial coordinate information by combining a maximum value searching algorithm and a minimum selection constant false alarm SO-CFAR algorithm;
the clustering module 130 is configured to cluster the point cloud data by adopting a DBSCAN clustering algorithm to obtain a clustering result;
the filtering module 140 is configured to filter interference information from the clustering result, where the interference information is obtained by scanning and determining an interfering object in the indoor environment by the millimeter wave radar;
and the tracking module 150 is used for tracking the clustering result with the interference information filtered by adopting Kalman filtering to obtain the tracking track of the indoor personnel.
The indoor personnel detection device in the embodiment of the present invention is used for executing the indoor personnel detection method in the above embodiment, and the specific processing procedure is the same as that of the indoor personnel detection method in the above embodiment, and will not be described in detail here.
In addition, as shown in fig. 13, an embodiment of the present invention further discloses an electronic device, including: at least one processor 210; at least one memory 220 for storing at least one program; the indoor person detection method as in any of the previous embodiments is implemented when the at least one program is executed by the at least one processor 210.
In addition, an embodiment of the present invention also discloses a computer-readable storage medium having stored therein computer-executable instructions for performing the indoor person detection method as in any of the previous embodiments.
The system architecture and the application scenario described in the embodiments of the present invention are for more clearly describing the technical solution of the embodiments of the present invention, and do not constitute a limitation on the technical solution provided by the embodiments of the present invention, and those skilled in the art can know that, with the evolution of the system architecture and the appearance of the new application scenario, the technical solution provided by the embodiments of the present invention is applicable to similar technical problems.
Those of ordinary skill in the art will appreciate that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.
In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
As used in this specification, the terms "component," "module," "system," and the like are intended to refer to a computer-related entity, either hardware, firmware, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, an object, an executable, a thread of execution, a program, or a computer. By way of illustration, both an application running on a computing device and the computing device can be a component. One or more components may reside within a process or thread of execution and a component may be localized on one computer or distributed between two or more computers. Furthermore, these components can execute from various computer readable media having various data structures stored thereon. The components may communicate by way of local or remote processes such as in accordance with a signal having one or more data packets (e.g., data from two components interacting with one another in a local system, distributed system, or across a network such as the internet with other systems by way of the signal).

Claims (10)

1. An indoor personnel detection method, comprising:
sampling and quantizing echo signals of personnel in the millimeter wave radar detection room through an analog-to-digital converter to obtain time domain signals;
Carrying out digital signal processing on the time domain signal, and generating point cloud data with space coordinate information by combining a maximum value searching algorithm and a minimum selection constant false alarm SO-CFAR algorithm;
clustering the point cloud data by adopting a DBSCAN clustering algorithm to obtain a clustering result;
filtering interference information from the clustering result, wherein the interference information is obtained by scanning and determining an interfering object in an indoor environment by the millimeter wave radar;
and tracking the clustering result of which the interference information is filtered by adopting Kalman filtering to obtain the tracking track of the indoor personnel.
2. The method according to claim 1, wherein the method further comprises:
determining a judging area used for judging whether indoor personnel enter or leave, a first loiter area located indoors and a second loiter area located outdoors in an indoor environment, wherein the judging area is located between a first mixing line and a second mixing line, the first mixing line is located between the first loiter area and the judging area, and the second mixing line is located between the judging area and the second loiter area;
under the condition that the tracking track of the indoor personnel sequentially passes through the first mixing line and the second mixing line, the number of people in the indoor personnel is counted and subtracted by one;
And under the condition that the tracking track of the indoor personnel sequentially passes through the second mixing line and the first mixing line, adding one to the statistics of the number of people in the indoor personnel.
3. The method of claim 1, wherein the point cloud data includes linear distance information, velocity information, energy information, azimuth angle information, and pitch angle information, wherein the performing digital signal processing on the time domain signal generates the point cloud data with space coordinate information by combining a maximum value search algorithm and a minimum selection constant false alarm SO-CFAR algorithm, and comprises:
performing distance Fourier transform and Doppler Fourier transform on the time domain signal sequentially to generate a distance Doppler two-dimensional matrix, wherein the distance Doppler two-dimensional matrix comprises a distance dimension and a Doppler dimension;
determining linear distance information from a target point to a millimeter wave radar in the indoor environment according to the index value of the distance dimension;
determining the speed information of the target point according to the index value of the Doppler dimension;
obtaining energy information of the target point by taking a model of the distance Doppler two-dimensional matrix result;
and searching the number of the target points and corresponding subscripts by combining a maximum value searching algorithm and a minimum selection constant false alarm SO-CFAR algorithm, performing Fourier transformation according to the antenna dimension of the millimeter wave radar, and obtaining azimuth angle information and pitch angle information of the target points through coordinate transformation.
4. The method according to claim 1, wherein the method further comprises:
the millimeter wave radar scans the indoor environment for a plurality of times to obtain newly added interference point cloud data;
storing the newly added interference point cloud data to a new cluster, and comparing the newly added interference point cloud data with interference information pre-stored in a Flash memory of the millimeter wave radar to obtain a comparison result;
under the condition that the comparison result is that the newly added interference point cloud data is similar to the interference information, the newly added interference point cloud data is stored in the Flash memory;
and under the condition that the comparison result is that the newly added interference point cloud data is dissimilar to the interference information, continuing to scan the indoor environment for the next time until the scanning frequency threshold is reached.
5. The method of claim 1, wherein the tracking the clustering result with the interference information filtered by kalman filtering to obtain the tracking track of the indoor personnel comprises:
estimating the state of the tracking group centroid at time n based on the state of time n-1 and a process noise covariance matrix by using a prediction process of Kalman filtering;
In the event that it is determined that there are one or more tracks and associated predicted state vectors, calculating a normalized distance function as a cost function for observations within a Gate of predicted centroids to associate an observation with each track, and minimizing the cost function during the allocation process, each time allocating an observation to the nearest track, allocating a series of point clouds for each tracked track;
and when the measured value at the moment n is obtained, updating the prediction state vector and the prediction error covariance estimation through Kalman filtering to obtain the tracking track of the indoor personnel.
6. The method of claim 5, wherein the method further comprises:
for observations outside any existing Gate, a new group tracker will be assigned and initialized;
selecting a forefront observation value, setting a centroid equal to the forefront observation value, screening other candidate observation values through the radial speed of the forefront observation value, recalculating the centroid under the condition that the candidate observation values are checked to be in a speed range, and adding the candidate observation values into the tracking group;
If it is determined that the number of observations in the tracking group meets a preset number threshold, the combined signal-to-noise ratio SNR is higher than the preset SNR threshold, and the dynamic measure of the centroid meets a preset measure threshold, a new tracking object is allocated and the discrete matrix is initialized using the associated observations in the tracking group.
7. The method of any one of claims 1 to 6, wherein the millimeter wave radar is a 60GHz frequency modulated continuous wave millimeter wave radar, the distance resolution is 5cm, the speed resolution is 0.1m/s, and the millimeter wave radar uses a 4x4 MIMO array antenna.
8. An indoor personnel detection apparatus, the apparatus comprising:
the sampling module is used for sampling and quantizing echo signals of personnel in the millimeter wave radar detection room through the analog-to-digital converter to obtain time domain signals;
the generation module is used for carrying out digital signal processing on the time domain signal, and generating point cloud data with space coordinate information by combining a maximum value search algorithm and a minimum selection constant false alarm SO-CFAR algorithm;
the clustering module is used for clustering the point cloud data by adopting a DBSCAN clustering algorithm to obtain a clustering result;
The filtering module is used for filtering interference information from the clustering result, wherein the interference information is obtained by scanning and determining an interfering object in an indoor environment by the millimeter wave radar;
and the tracking module is used for tracking the clustering result with the interference information filtered by adopting Kalman filtering to obtain the tracking track of the indoor personnel.
9. An electronic device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the indoor person detection method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing computer-executable instructions for performing the indoor person detection method according to any one of claims 1 to 7.
CN202311494857.9A 2023-11-09 2023-11-09 Indoor personnel detection method and device, electronic equipment and storage medium Pending CN117687011A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311494857.9A CN117687011A (en) 2023-11-09 2023-11-09 Indoor personnel detection method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311494857.9A CN117687011A (en) 2023-11-09 2023-11-09 Indoor personnel detection method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN117687011A true CN117687011A (en) 2024-03-12

Family

ID=90136128

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311494857.9A Pending CN117687011A (en) 2023-11-09 2023-11-09 Indoor personnel detection method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117687011A (en)

Similar Documents

Publication Publication Date Title
US11435443B2 (en) Integration of tracking with classifier in mmwave radar
Konstantinova et al. A study of a target tracking algorithm using global nearest neighbor approach.
WO2008063946A2 (en) Method and system for automatically estimating the spatial positions of cameras in a camera network
US11041947B2 (en) Radar detection methods and systems for identifying moving objects
Jana et al. Compensation of wall effect for through wall tracking of moving targets
CN112731366A (en) Indoor personnel positioning method, device and system
CN116125458A (en) Personnel positioning method based on millimeter wave radar
Arsan et al. A Clustering‐Based Approach for Improving the Accuracy of UWB Sensor‐Based Indoor Positioning System
Ma et al. Device-free, activity during daily life, recognition using a low-cost lidar
Xue et al. A machine learning-based positioning method for poultry in cage environments
CN117687011A (en) Indoor personnel detection method and device, electronic equipment and storage medium
Bartoletti et al. Positioning Methods
KR100811887B1 (en) Apparatus and method for providing selectively position information having steps accuracy in autonomous mobile robot
Yu et al. An intelligent space location identification system based on passive RFID tags
Jung et al. Localization based on distributed heterogeneous radar sensor network
Li et al. Indoor Multi-Human Device-Free Tracking System Using Multi-Radar Cooperative Sensing
Canil et al. Oracle: Occlusion-resilient and self-calibrating mmwave radar network for people tracking
JP2022001864A (en) Method, device and electronic apparatus for detecting moving object
Alba et al. Uncertainty-boosted radiomap-based indoor positioning with RSSI fingerprints
Woznica et al. RF indoor positioning system supported by wireless computer vision sensors
Haritha et al. Investigation on Indoor Detection and Tracking of People using IWR1642 mmWave Sensor
Li et al. Counting and Tracking People to Avoid from Crowded in a Restaurant Using mmWave Radar
TWI767601B (en) Electronic device and method for indoor positioning, imaging, detecting, posture discrimination, and shape discrimination
Lu et al. Learning-Based Cognitive Radar Resource Management for Scanning and Multi-Target Tracking
KR20190019756A (en) Method and system for determining location using Newton iteration

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination