CN113017590A

CN113017590A - Physiological data monitoring method and device, computer equipment and storage medium

Info

Publication number: CN113017590A
Application number: CN202110214267.0A
Authority: CN
Inventors: 赵自然; 张永申; 乔灵博
Original assignee: Tsinghua University
Current assignee: Tsinghua University
Priority date: 2021-02-26
Filing date: 2021-02-26
Publication date: 2021-06-25
Anticipated expiration: 2041-02-26
Also published as: CN113017590B

Abstract

The application relates to a physiological data monitoring method, a physiological data monitoring device, a computer device and a storage medium. The method comprises the following steps: collecting a packet image in a monitoring area by utilizing camera equipment; determining three-dimensional position information of a designated body part of a walking person in the image by using an image target detection and positioning algorithm; calculating the distance between the appointed body part of the walking person in the image and the radar in a non-fluctuation state by utilizing the three-dimensional position information and determining the transmitting direction of the radar main beam; acquiring a dynamic echo signal received by a radar; converting the distance between the specified body part of the walking person and the radar in a non-fluctuating state to obtain reference time; calculating a static echo signal according to the reference time and the dynamic echo signal; physiological data of a walking person is extracted from the static echo signals. By adopting the method, the physiological data of the walking person can be monitored in a non-contact manner.

Description

Physiological data monitoring method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of physiological data monitoring technology, and in particular, to a physiological data monitoring method, apparatus, computer device, and computer-readable storage medium.

Background

The heart rate and the respiration rate are important physiological data for judging the health condition, emotion and the like of a human body, respiration information is measured by a respiration belt and heart rate information is measured by an electrode electrocardiogram in the prior art, but the contact measurement modes cannot adapt to the flowing places of people. With the development of contactless respiratory rate and heart rate measurement technical means in recent years, the measurement results of contactless technologies including continuous wave doppler radar, frequency modulation continuous wave radar, pulse ultra-wideband radar and WiFi are very close to the standard contact measurement results. However, the above-mentioned techniques require that the measured object is in a static state, and the measurement result is biased when the human body has a small movement.

Disclosure of Invention

In view of the above, there is a need to provide a physiological data monitoring method, apparatus, computer device and computer readable storage medium capable of contactless measurement of a walking person.

In a first aspect, a method for monitoring physiological data is provided, the method comprising:

collecting images in a monitoring area by utilizing camera equipment; when only one walking person is allowed to pass through the monitoring area in one way at the same time, determining whether the walking person exists in the monitoring area in the image or not by utilizing an image target detection and positioning algorithm aiming at each acquired frame of image, and if the walking person exists in the monitoring area, determining the three-dimensional position information of the appointed body part of the walking person by utilizing the image target detection and positioning algorithm; calculating the distance between the designated body part and the radar according to the three-dimensional position information of the designated body part and the three-dimensional position information of the radar to obtain the distance between the designated body part of the walking person in each frame of image and the radar in a non-fluctuating state; determining the transmitting direction of a radar main beam according to the three-dimensional position information of the designated body part in each frame of image, the mounting position of the radar and the mounting posture of the radar, and controlling the radar to transmit electromagnetic waves to the transmitting direction of the radar main beam in a preset working mode; the preset working mode comprises a pulse ultra-wideband mode or a frequency modulation continuous wave mode; acquiring dynamic echo signals received by the radar under the fluctuation state of the appointed body part of a walking person in each frame of image; converting the distance between the specified body part of the walking person in each frame of image and the radar in a non-fluctuating state to obtain the reference time of the electromagnetic waves from the radar to the specified body part and then returning to the radar in the non-fluctuating state of the specified body part of the walking person; calculating to obtain a static echo signal of an equivalent static human body part corresponding to each frame of image in the fluctuating state according to the reference time corresponding to each frame of image and the dynamic echo signal; and extracting physiological data of the walking person from a plurality of static echo signals corresponding to the multi-frame images.

In one embodiment, the calculating the static echo signal under the fluctuating state of the specified body part of the equivalent static person corresponding to each frame of image according to the reference time and the dynamic echo signal corresponding to each frame of image comprises:

when the working mode of the radar is pulse ultra-wideband, acquiring a dynamic echo signal of an ith frame of image in the image corresponding to the fluctuation state of the appointed body part

For the dynamic echo signal

Fourier transform and exp (j ω t) multiplication_i) And then carrying out Fourier inversion to obtain a static echo signal in an up-and-down state of the equivalent static human designated body part corresponding to the ith frame image

Where j is the unit of an imaginary number, ω is the angular frequency, t_iAnd the reference time of the electromagnetic wave from the radar to the specified body part and then returning to the radar in the state that the specified body part of the walking person corresponding to the image i is not fluctuated.

when the working mode of the radar is frequency modulation continuous wave, acquiring a dynamic echo signal of an ith frame of image in the image corresponding to the fluctuation state of the appointed body part

For the dynamic echo signal

After two Fourier transforms, multiply by exp (j ω t)_i) And then carrying out Fourier inversion to obtain a static echo signal corresponding to the image i under the fluctuation state of the equivalent static human designated body part

In one embodiment, when the radar is a phased array radar, the method further comprises:

when a plurality of walking persons are allowed to pass through the monitoring area in one way at the same time, determining whether the walking persons exist in the monitoring area or not by utilizing an image target detection and positioning algorithm aiming at each acquired frame image, and if the walking persons exist in the monitoring area, determining the head image of each walking person and the three-dimensional position information of the appointed body part of the walking person in each frame image by utilizing the image target detection and positioning algorithm; matching the head image of each walking person with the head images of the existing walking persons by using a matching algorithm, and creating a label for each walking person entering a monitoring area for the first time according to a matching result; the static echo signal of each walking person is recorded under the corresponding tag.

In a second aspect, a method of physiological data monitoring is provided, the method comprising: monitoring physiological data of a walking person by using n monitoring devices arranged in the monitoring area when only one walking person is allowed to pass through the monitoring area at the same time, wherein n is an integer greater than or equal to 2; acquiring images in a monitoring area through a first monitoring device, determining whether a walking person exists in the monitoring area in the images by using an image target detection and positioning algorithm aiming at each acquired frame of image through an image analysis module of the first monitoring device, and if the walking person exists, acquiring a static echo signal of an equivalent static person corresponding to the acquired images in the first monitoring area under the fluctuation state of an appointed body part; when the walking person walks out of the first monitoring subarea, the first monitoring device sends a static echo signal to the second monitoring device; the second monitoring device obtains a static echo signal of the appointed body part of the equivalent static person corresponding to the collected image in the second monitoring subarea in the fluctuating state; when the walking person walks out of the second monitoring subarea, the second monitoring device sends the static echo signal corresponding to the first monitoring device and the static echo signal corresponding to the second monitoring device to a third monitoring device; until the walking person walks out of the nth-1 monitoring subarea, the nth-1 monitoring device sends the static echo signals corresponding to the first monitoring device to the nth-1 monitoring device; the nth monitoring device obtains static echo signals of the equivalent static human body part in the fluctuating state corresponding to the collected image in the nth subarea, and the data processing module of the nth monitoring device extracts the physiological data of the walking human body from the static echo signals corresponding to the first to nth monitoring devices.

In one embodiment, the monitoring device comprises a camera device, a radar, an image analysis module and a data processing module; wherein, the method also comprises:

acquiring an image in a monitoring area through the camera equipment; when only one walking person is allowed to pass through the monitoring area in one direction at the same moment, aiming at each frame of image, determining the three-dimensional position information of the appointed body part of the walking person by using an image target detection and positioning algorithm through the image analysis module; calculating the distance between the designated body and the radar through the data processing module according to the three-dimensional position information of the designated body part and the three-dimensional position information of the radar to obtain the distance between the designated body part of the walking person in each frame of image and the radar in a non-fluctuating state; determining the transmitting direction of the radar main beam according to the three-dimensional position information of the designated body part in each frame of image, the mounting position of the radar and the mounting posture of the radar by an analysis module of the radar, and controlling the radar to transmit electromagnetic waves to the transmitting direction of the radar main beam in a preset working mode; the preset working mode comprises a pulse ultra-wideband mode or a frequency modulation continuous wave mode; acquiring dynamic echo information received by the radar under the fluctuation state of the appointed body part of a walking person in each frame of image through the data processing module; the distance between the designated body part of the walking person in each frame of image and the radar is converted through the data processing module under the non-fluctuating state, and the reference time of the electromagnetic waves from the radar to the designated body part and then returning to the radar under the non-fluctuating state of the designated body part of the walking person is obtained; and calculating to obtain a static echo signal of the equivalent static human body part corresponding to each frame of image in the fluctuating state according to the reference time and the dynamic echo signal corresponding to each frame of image by the data processing module.

In a third aspect, there is provided a physiological data monitoring device, the device comprising:

the image acquisition module is used for acquiring images in a monitoring area by utilizing the camera equipment;

the position determining module is used for determining whether a walking person is contained in the monitoring area in the acquired image or not by utilizing an image target detection and positioning algorithm aiming at each frame of image when only one walking person is allowed to pass through the monitoring area at the same moment, and if the walking person exists in the monitoring area, determining the three-dimensional position information of the appointed body part of the walking person by utilizing the image target detection and positioning algorithm;

the distance determining module is used for calculating the distance between the specified body part and the radar according to the three-dimensional position information of the specified body part and the three-dimensional position information of the radar to obtain the distance between the specified body part of the walking person in each frame of image and the radar in a non-fluctuating state;

the direction determining module is used for determining the transmitting direction of the radar main beam according to the three-dimensional position information of the designated body part in each frame of image, the mounting position of the radar and the mounting posture of the radar, and controlling the radar to transmit electromagnetic waves to the transmitting direction of the radar main beam in a preset working mode; the preset working mode comprises a pulse ultra-wideband mode or a frequency modulation continuous wave mode;

the signal acquisition module is used for acquiring dynamic echo signals received by the radar under the fluctuating state of the appointed body part of the walking person in each frame of image;

the transformation module is used for transforming the distance between the specified body part of the walking person in each frame of image and the radar in the non-fluctuating state to obtain the reference time of the electromagnetic waves from the radar to the specified body part and then returning to the radar in the non-fluctuating state of the specified body part of the walking person;

the motion compensation module is used for calculating and obtaining a static echo signal under the fluctuation state of the equivalent static human designated body part corresponding to each frame of image according to the reference time and the dynamic echo signal corresponding to each frame of image;

and the data extraction module is used for extracting the physiological data of the walking person from a plurality of static echo signals corresponding to the multi-frame images.

In a fourth aspect, there is provided a computer device comprising a memory storing a computer program and a processor implementing the physiological data monitoring method according to any one of the first aspect or the second aspect when the processor executes the computer program.

In a fifth aspect, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the physiological data monitoring method according to any one of the first aspect or the physiological data monitoring method according to any one of the second aspect.

According to the physiological data monitoring method, the physiological data monitoring device, the computer equipment and the storage medium, the three-dimensional position information of the designated body part of the walking person is obtained through the image, the three-dimensional position information is firstly used for calculating the distance between the designated body part of the walking person and the radar in a non-fluctuating state, and the distance reflects the reference position of the walking person; and secondly, the method is also used for calculating the angular position of the specified body part of the walking person relative to the radar and adjusting the emission direction of the electromagnetic wave of the radar. A dynamic echo signal in a state where a specified body part of a walking person is fluctuated is obtained by transmitting an electromagnetic wave by a radar and collecting an echo, the dynamic echo signal reflecting a reference position of the walking person and a movement of the specified body part relative to the person caused by the fluctuation of the specified body part. The reference position of the walking person is utilized to perform motion compensation on the reference position of the walking person and the movement of the appointed body part relative to the person caused by the fluctuation of the appointed body part, and a static echo signal only containing the movement information of the appointed body part relative to the person caused by the fluctuation of the appointed body part is obtained, so that the physiological data monitoring problem of the dynamic walking person is converted into the physiological data monitoring problem of the static person, and the accurate monitoring of the non-contact physiological data of the walking person is realized.

Drawings

FIG. 1 is a schematic illustration of an implementation environment in one embodiment;

FIG. 2 is a schematic diagram of a physiological data monitoring module in one embodiment;

FIG. 3 is a schematic diagram of another physiological data monitoring module in one embodiment;

FIG. 4 is a schematic flow chart diagram of a physiological data monitoring method in one embodiment;

FIG. 5 is a schematic diagram of an image captured by an imaging device in one embodiment;

FIG. 6 is a schematic diagram of an image captured by an imaging device in one embodiment;

FIG. 7 is a diagram illustrating a plurality of dynamic echo signals, in accordance with one embodiment;

FIG. 8 is a diagram illustrating a plurality of static echo signals, in accordance with one embodiment;

FIG. 9 is a schematic illustration of computing physiological data using static echoes in one embodiment;

FIG. 10 is a schematic flow chart of a physiological data monitoring method in another embodiment;

FIG. 11 is a schematic illustration of monitoring physiological data of a plurality of walking persons;

FIG. 12 is a schematic view of a plurality of monitoring devices deployed in a monitoring area for physiological data monitoring;

FIG. 13 is a block diagram of a physiological data monitoring device in one embodiment;

FIG. 14 is a block diagram of a physiological data monitoring device in accordance with another embodiment;

FIG. 15 is a block diagram of a physiological data monitoring device in accordance with another embodiment;

FIG. 16 is a system framework of a physiological data monitoring device in one embodiment;

FIG. 17 is a diagram illustrating an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

As shown in fig. 1, the environment in which the physiological data monitoring method is implemented may include an image capturing device 102, a radar 104, an image analysis module 106, and a data processing module 108. The functions of the image analysis module 106 and the data processing module 108 may be implemented by a processor. In addition, the camera device and the radar may be integrated into one physiological data monitoring module, fig. 2 and 3 show schematic diagrams of two physiological data monitoring modules, the camera device and the radar are integrated into one physiological data monitoring module, the image analysis module and the data processing module are located at the rear end of the physiological data monitoring module, and the image analysis module and the data processing module are not shown in fig. 2 and 3. In fig. 2, the radar 204 is a mechanical rotation radar, and in fig. 3, the radar 304 is a phased array radar. Mechanical rotation radar refers to radar that relies on the rotation of a radar antenna to achieve beam scanning, and in one embodiment of the present application, the mechanical rotation radar may be mounted on a structure that is capable of rotation in two dimensions. The phased array radar realizes beam scanning in a phased array mode, the phase of each array element required for enabling the synthesized beam to point to a target direction can be calculated according to an array arrangement mode and an antenna array element directional diagram, and electromagnetic waves can be transmitted to the specified target direction after phase adjustment is completed.

In one embodiment, as shown in fig. 4, there is provided a physiological data monitoring method comprising the steps of:

and step 402, acquiring an image in a monitored area by using a camera.

In one embodiment, the monitoring area is a preset range, and the direction from the starting point to the end point of the monitoring area is the walking direction of the pedestrian. The camera equipment comprises at least one of a camera and a thermal infrared imager, wherein the thermal infrared imager can be used for image acquisition when light rays are weak at night. In one embodiment of the application, the camera device is placed at the end point of the monitoring area, the camera device can continuously acquire images at a preset constant frame rate, the camera device transmits the images to the image processing module, and the image processing module determines images of people walking in the monitoring area by using an image target detection and positioning algorithm; fig. 5 and 6 are schematic diagrams of images acquired by the camera device. Each frame of image collected by the camera equipment is immediately transmitted into the image analysis module, and the image collection and the transmission of each frame of collected image into the image analysis module are synchronously carried out.

Step 404, when only one walking person is allowed to pass through the monitoring area in one way at the same time, determining whether the walking person exists in the monitoring area or not by using an image target detection and positioning algorithm aiming at each acquired frame image, and if the walking person exists in the monitoring area, determining the three-dimensional position information of the appointed body part of the walking person by using the image target detection and positioning algorithm.

In an optional embodiment of the application, the traffic policy defines that only one pedestrian is allowed to pass through the monitoring area in one direction at the same time, that is, only one walking person is allowed to pass through the monitoring area from the starting point of the monitoring area to the end point of the monitoring area at the same time, and the passing direction of the walking person is the same as the direction from the starting point of the monitoring area to the end point of the monitoring area. The image analysis module receives an image transmitted by the camera equipment, judges whether a walking person exists in a monitoring area by using an image target detection and positioning algorithm, if the walking person exists, the deep learning model outputs three-dimensional position information of a specified body part of the walking person, and the image analysis module transmits the three-dimensional position information of the specified body part to the radar and data processing module. If no walking person exists in the image, the image analysis module does not transmit any information to the data processing module and the radar.

In an alternative embodiment of the present application, the designated body part may be the human's chest. The image target detection and positioning algorithm based on the deep learning model mainly realizes two functions, namely firstly, detecting whether a walking person exists in a monitoring area, and secondly, outputting three-dimensional position information of the chest center position of the walking person if the walking person is located in the monitoring area. Whether walking people exist in the monitoring area or not is detected, and the first condition is to judge whether the walking people enter the monitoring area or not, and the judgment can be carried out according to the position relation between the walking people in the image and the starting point of the monitoring area. The other method is to judge whether the walking person walks out of the monitoring area, and because the camera device is placed at the end point of the monitoring area, when the images transmitted by the camera device continuously contain the walking person in the monitoring area, and the subsequent transmitted images no longer contain the walking person in the monitoring area, the walking person can be known to have walked out of the monitoring area. When walking people exist in a monitoring area, firstly, a deep learning model needs to be trained, before the deep learning model is trained, the position of a camera device is fixed, a marker is selected, the people are located at different positions of the monitoring area and hold breath, and a large number of image samples are obtained by shooting through the camera device. And (3) preprocessing the image samples, and measuring the three-dimensional position information of the center position of the chest of the person in each frame of image sample through experiments. When the deep learning model is trained, image samples are input, and the label corresponding to each frame of image sample is three-dimensional position information of the center position of the human prothorax. After the deep learning model training is finished, the position and the marker of the camera equipment are not changed, the image analysis module inputs a frame of image acquired by the camera equipment into an image target detection and positioning algorithm based on the deep learning model, when the algorithm carries out prediction, whether a walking person exists in the image or not is judged firstly, and if the walking person does not exist in the monitoring area, no output is carried out; and if the walking person exists in the monitoring area, outputting three-dimensional position information of the center position of the chest of the walking person by the algorithm. The image sample obtains three-dimensional position information of the center position of the chest when the person holds the breath, and the chest is in a non-fluctuation state when the person holds the breath, wherein the non-fluctuation state means that the chest does not contract or expand. Therefore, when the three-dimensional position information of the central position in the non-fluctuating state of the chest is used as a training sample to train the deep learning model and the deep learning model is used for prediction, the output result of the deep learning model is the prediction result of the three-dimensional position information of the central position in the non-fluctuating state of the chest.

And step 406, calculating the distance between the designated body part and the radar according to the three-dimensional position information of the designated body part and the three-dimensional position information of the radar, and obtaining the distance between the designated body part of the walking person in each frame of image and the radar in a non-fluctuating state.

As described above, the three-dimensional position information of the specified body part center position of the walking person is determined with respect to the selected marker, and in an alternative embodiment of the present application, the three-dimensional position information of the radar is determined based on the relative positions of the marker and the radar after the radar is installed. Transmitting the three-dimensional position information of the radar into a data processing module, and calculating by the data processing module as follows:

suppose that the three-dimensional position information of the specified body part center position of the person walking in the image with number i is (x)_1i,y_1i,z_1i) The three-dimensional position information of the radar is (x)₂,y₂,z₂) Then, the distance d between the designated body part corresponding to the image with the sequence number i and the radar_iComprises the following steps:

as mentioned above, since the image sample obtains the three-dimensional position information of the center position of the chest when the person holds his/her breath, the chest is in a non-fluctuating state when the person holds his/her breath, and the non-fluctuating state means that the chest is not contracted or expanded. Therefore, when the three-dimensional position information of the central position in the non-fluctuating state of the chest is used as a training sample to train the deep learning model and the deep learning model is used for prediction, the output result of the deep learning model is the prediction result of the three-dimensional position information of the central position in the non-fluctuating state of the chest. And calculating the distance between the specified body part of the walking person and the radar in a non-fluctuating state by using the three-dimensional position information of the specified body part of the walking person and the three-dimensional position information of the radar.

Step 408, determining the transmitting direction of the radar main beam according to the three-dimensional position information of the specified body part, the installation position of the radar and the installation posture of the radar, and controlling the radar to transmit electromagnetic waves to the transmitting direction of the radar main beam in a preset working mode; the preset working mode comprises a pulse ultra-wideband mode or a frequency modulation continuous wave mode.

When obtaining a dynamic echo signal in a state where a specified body part of a walking person is fluctuated, it is necessary for a radar to emit an electromagnetic wave to the specified body part of the walking person, and since the person is moving, it is necessary to constantly adjust the direction in which the radar emits the electromagnetic wave. The transmitting direction is the direction of the specified body part of the walking person compared with the radar, and as described above, the three-dimensional position information of the specified body part of the walking person can be obtained through the image target detection and positioning algorithm based on the deep learning model. In an optional embodiment of the application, the image analysis module sends the three-dimensional position information of the specified body part to the radar analysis module, the radar analysis module can calculate the angular position of the specified body part of the walking person in the image relative to the radar by combining the installation position and the installation posture of the radar, and the direction of the electromagnetic wave transmitted by the radar is adjusted according to the angular position. After the adjustment of the transmitting direction is completed, electromagnetic waves are transmitted to the transmitting direction in one mode of a pulse ultra-wideband mode or a frequency modulation continuous wave mode, the image analysis module sends the three-dimensional position information of the specified body part of the walking person in each frame of image to the analysis module of the radar, the radar analysis module adjusts the direction of transmitting the electromagnetic waves and the transmitting process according to the three-dimensional position information very quickly, and when the electromagnetic waves reach the specified body part of the walking person in the monitoring area, the position of the walking person in the monitoring area is not displaced compared with the position of the walking person in the monitoring area when the frame of image is shot.

And step 410, acquiring dynamic echo signals received by the radar in a fluctuating state of the appointed body part of the walking person in each frame of image.

As described above, the radar adjusts the direction and transmits the electromagnetic wave according to the three-dimensional position information of the specified body part of the person walking in each frame image. In an alternative embodiment of the present application, the radar receives dynamic echo signals returned after electromagnetic waves reach a designated body part of a walking person. In an alternative embodiment of the present application, the designated body part may be a chest, the rolling state refers to contraction or expansion of the chest caused by respiration or heart beating of the person, when the designated body part of the walking person is in the rolling state, the electromagnetic wave can accurately detect the micro contraction or expansion of the designated body part, and therefore the dynamic echo signal refers to an echo signal received after the radar transmits the electromagnetic wave when the designated body part of the walking person rolls. The radar transmits the dynamic echo signal to a data processing module.

And step 412, converting the distance between the specified body part of the walking person in each frame of image and the radar in the non-fluctuating state to obtain the reference time of the electromagnetic waves from the radar to the specified body part and then returning to the radar in the non-fluctuating state of the specified body part of the walking person.

Since the radar echo signal is a curve with time as a variable, it is not possible to directly use the distance from the radar in the undulated state of the designated body part of the walking person to perform motion compensation on the dynamic echo signal in the undulated state of the designated body part of the walking person, and therefore, it is necessary to first convert the distance from the radar in the undulated state of the designated body part of the walking person.

In an optional embodiment of the present application, the data processing module receives the distance d from the radar when the designated body part of the walking person corresponding to the ith frame of image is in a non-fluctuating state_iThe distance is divided by the propagation speed of the electromagnetic wave to obtain the time of the electromagnetic wave from the radar to the specified body part of the walking person, the time is 2 times of the reference time required by the electromagnetic wave from the radar to the specified body part of the walking person and then returning to the radar, and the reference time is stored in the data processing module.

Wherein i is the serial number of the image; t is t_iThe time of the electromagnetic wave from the radar to the specified body part and then returning to the radar under the state that the specified body part of the walking person is not fluctuated; d_iThe distance between the specified body part of the walking person and the radar in a non-fluctuating state; and c is the propagation velocity of the electromagnetic wave.

And step 414, calculating to obtain the static echo signal of the walking person in the fluctuating state of the appointed body part corresponding to each frame of image according to the reference time corresponding to each frame of image and the dynamic echo signal.

As described above, the radar emits electromagnetic waves to the specified body part of the walking person, and the motion echo signal corresponding to the i-th frame image accurately reflects the reference position of the walking person and the movement of the specified body part relative to the person due to the fluctuation of the specified body part. The distance from the radar in the state where the designated body part of the walking person is free from undulation is calculated from the i-th frame image, and since the designated body part does not move relative to the person when the designated body part is free from undulation, the distance reflects the reference position of the walking person.

In an optional embodiment of the present application, since the dynamic echo signal takes time as a variable, the distance reflecting the reference position of the walking person is converted into a reference time, the data processing module performs motion compensation on the dynamic echo signal by using the reference time, that is, the reference position of the walking person is used to perform motion compensation on the reference position of the walking person and the movement of the specified body part relative to the person caused by the fluctuation of the specified body part, the influence of the reference position of the walking person on the monitoring of the physiological data is eliminated, and a static echo signal in the fluctuation state of the specified body part of the walking person corresponding to each frame image is obtained, and the static echo signal only reflects the movement of the specified body part relative to the person. As shown in fig. 7, the received dynamic echo signals are stored in columns, the dynamic echo signal received for the first time is placed in the first column, the dynamic echo signal received for the second time is placed in the second column, and so on. In the method, the radar emits electromagnetic waves once every time a frame of image is received, so that the slow time can be represented by an image sequence number, and the ordinate is the fast time. As shown in fig. 8, the influence of the human body reference position information included in each row of the dynamic echo signals in fig. 7 on the physiological data monitoring is eliminated by using the reference time to obtain static echo signals, and the static echo signals are stored in rows, wherein the abscissa is the slow time represented by the image sequence number, and the ordinate is the fast time.

Step 416, the physiological data of the walking person is extracted from the plurality of static echo signals corresponding to the plurality of frames of images.

In an alternative embodiment of the present application, the physiological data may include heart rate and respiration rate. The number of image frames shot by the camera equipment in the time from the walking person entering the monitoring area to the walking person leaving the monitoring area is the number of multi-frame images, and the specific number of the multi-frame images is related to the walking speed of the walking person.

In one embodiment of the present application, the static echo signals are stored in columns, as described above, and a matrix with slow time on the abscissa and fast time on the ordinate can be obtained. The camera shooting equipment collects n frames of images of a walking person in a monitoring area and respectively corresponds to n static echo signals. Wherein the first static echo signal to the mth static echo signal are grouped, the second static echo signal to the m +1 static echo signal are grouped, and so on, the n-m +1 static echo signal to the nth static echo signal are grouped, wherein m < n. Each group being a small matrix. Calculating the variance of each row of each small matrix, determining the row with the largest variance, enabling the data of the row with the largest variance to pass through a high-pass filter to obtain a heartbeat signal, and performing Fourier transform on the heartbeat signal to obtain the frequency corresponding to the amplitude peak value, namely the heart rate; and (4) passing the data of the row with the maximum variance through a low-pass filter to obtain a respiratory signal, and performing Fourier transform on the respiratory signal to obtain the frequency corresponding to the amplitude peak value, namely the respiratory rate. The total number of n-m +1 groups of static echo signals can be calculated to obtain n-m +1 heart rates and n-m +1 respiration rates, different heart rates and respiration rates represent physiological data of the walking person at different moments, and a heart rate sequence and a respiration rate sequence are obtained.

In a specific embodiment, as shown in fig. 9, each column represents a static echo signal of a walking person of physiological data to be monitored, the image capturing device acquires seven frames of images of the walking person in a monitoring area, the seven static echo signals correspond to the walking person, a first static echo signal to a fifth static echo signal are a first group, a second static echo signal to a sixth static echo signal are a second group, and a third static echo signal to a seventh static echo signal are a third group, and a heart rate 1, a respiratory rate 1, a heart rate 2, a respiratory rate 2, a heart rate 3, and a respiratory rate 3 can be respectively calculated according to the three groups of static echo signals.

In the physiological data monitoring method, three-dimensional position information of the designated body part of the walking person is obtained through the image, the three-dimensional position information is firstly used for calculating the distance between the designated body part of the walking person and the radar in a non-fluctuating state, and the distance reflects the reference position of the walking person; and secondly, the method is also used for calculating the angular position of the radar relative to the specified body part of the walking person and adjusting the emission direction of the electromagnetic wave of the radar. The method includes the steps of transmitting an electromagnetic wave by a radar and collecting an echo to obtain a dynamic echo signal of a walking person in a state where a designated body part fluctuates, the dynamic echo signal reflecting a reference position of the walking person and a movement of the designated body part relative to the person due to fluctuation of the designated body part. The reference position of the walking person is utilized to perform motion compensation on the reference position of the walking person and the movement of the appointed body part relative to the person caused by the fluctuation of the appointed body part, and a static echo signal only containing the movement information of the appointed body part relative to the person caused by the fluctuation of the appointed body part is obtained, so that the physiological data monitoring problem of the dynamic walking person is converted into the physiological data monitoring problem of the static person, and the accurate monitoring of the non-contact physiological data of the walking person is realized.

In one embodiment, the calculating the static echo signal under the fluctuation state of the equivalent static human designated body part corresponding to each frame of image according to the reference time corresponding to each frame of image and the dynamic echo signal comprises:

For the dynamic echo signal

When the working mode of the radar is the pulse ultra-wideband, the influence of the human body reference position in the dynamic echo signal is eliminated by utilizing the reference time, and the static echo signal corresponding to each frame of image under the fluctuation state of the equivalent static human designated body part is obtained.

For the dynamic echo signal

When the working mode of the radar is frequency modulation continuous wave, the influence of the human body reference position in the dynamic echo signal is eliminated by utilizing the reference time, and the static echo signal corresponding to each frame of image under the fluctuation state of the equivalent static human designated body part is obtained.

In one embodiment, as shown in fig. 10, the radar is a phased array radar, and the physiological data monitoring method further includes:

step 502, when a plurality of walking persons are allowed to pass through the monitoring area in one direction at the same time, determining whether the walking persons exist in the monitoring area or not by using an image target detection and positioning algorithm aiming at each acquired frame image, and if the walking persons exist in the monitoring area, determining the head image of each walking person and the three-dimensional position information of the appointed body part of the walking person in each frame image by using the image target detection and positioning algorithm.

In one embodiment of the present application, as described above, the image target detection and positioning algorithm can achieve two functions, the first is to detect whether there is a walking person in the monitoring area, and the second is to output three-dimensional position information of the chest center position of the walking person if there is a walking person in the monitoring area. In addition to the above embodiments, the image target detection and positioning algorithm based on the deep learning model can also output the head image of the walking person when the walking person is located in the monitoring area. The method for acquiring the image sample is not changed, when the image is preprocessed, the head image of each walking person in the image is manually marked, and the three-dimensional position information of the center position of the chest of the walking person in the image is measured through experiments. When the deep learning model is trained, image samples are input, a first training label corresponding to each frame of image sample is a head image of a walking person in the frame of image sample, and a second training label corresponding to each frame of image sample is three-dimensional position information of the center position of the chest of the walking person. After the deep learning model training is completed, inputting a frame of image into the deep learning model, firstly judging whether a walking person is in a monitoring area or not in the image by the deep learning model, if the walking person is not in the monitoring area, the deep learning model has no output, and if the walking person is in the monitoring area, the deep learning model outputs three-dimensional position information of the chest center position of the walking person and a data pair of the head image of the walking person.

And step 504, matching the head image of each walking person with the head images of the walking persons by using a matching algorithm, and creating a label for each walking person entering the monitoring area for the first time according to a matching result.

The image analysis module stores head images of walking people and corresponding labels thereof. When the image analysis module obtains the head image of each walking person in the image and the three-dimensional position information of the designated body part by using the image target detection and positioning algorithm, the image analysis module will transfer the head images of all walking people in one frame of image into the matching algorithm, for the head image of each walking person in one frame of image, the matching algorithm respectively compares the head image of the walking person with the head image of each walking person stored in the image analysis module, if the head image of the walking person existing in the image analysis module can not be matched, the image pickup device is used for shooting the walking person for the first time in the monitoring area, the matching algorithm creates a label for the walking person, establishes mapping between the head image of the walking person and the label, and stores the head image of the walking person and the mapping relation between the head image of the walking person and the label in an image analysis module; if the head image of the walking person existing in the image analysis module can be matched, the fact that the image of the walking person is shot by the previous camera device is shown, and the walking person is still in the monitoring area is shown, the matching algorithm allocates a label corresponding to the head image of the walking person existing to the walking person, and mapping between the head image of the walking person and the label is established; if the head image of the existing walking person in the image analysis module cannot be matched with the head image of each walking person in one frame of image, the walking person corresponding to the head image of the existing walking person is shown to have already walked out of the monitoring area, the monitoring process of the walking person is completed, and the head image of the existing walking person and the corresponding label are deleted in the image analysis module.

Step 506, the static echo signal of each walking person is recorded under the corresponding tag.

In the embodiment of the application, when the radar is a phased array radar, electromagnetic waves with a plurality of beams can be transmitted at the same time, so that the phased array radar can be used for monitoring physiological data of a plurality of walking persons at the same time. As described above, the image analysis module recognizes the head images of a plurality of walking persons in each frame image by using a matching algorithm to obtain a tag corresponding to the head image of each walking person, stores the three-dimensional position information of the designated body part of the walking person under the tag corresponding to the head image of the walking person, and transmits the tag in which the three-dimensional position information of the designated body part is stored to the phased array radar and data analysis module. The data analysis module stores a reference time calculated from the three-dimensional position information of the specified body part under the tag. The phased array radar transmits electromagnetic waves with a plurality of wave beams at the same time based on three-dimensional position information of designated body parts corresponding to different tags, obtained dynamic echo signals are stored under the corresponding tags respectively, and the dynamic echo signals stored under the corresponding tags are transmitted to the data analysis module. And the data analysis module calculates static echo signals under the same label by using the reference time and the dynamic echo signals under the same label, wherein each label corresponds to a walking person.

In this embodiment, the head image of each walking person in the image is matched with the walking person existing in the image analysis module by using a matching algorithm, and a label is created for each walking person entering the monitoring area for the first time according to a matching result. When the radar is a phased array radar, electromagnetic waves with a plurality of wave beams can be emitted at the same time, physiological data of a plurality of walking people in a monitoring area are monitored, and a plurality of static echo signals obtained by the data analysis module are distinguished by the label. The method can utilize one phased array radar to realize physiological data monitoring on a plurality of walking persons in a monitoring area, and realize more efficient monitoring of the physiological data of the walking persons.

In a specific embodiment of the present application, as shown in fig. 11, the image capturing apparatus continues to capture images as time advances, and when capturing a first frame of image, a first walking person walks into the monitoring area, which is referred to as a target 1; when the fifth frame image is shot, a second walking person walks into the monitoring area, and the second walking person is called a target 2; the camera equipment transmits the acquired images to an image analysis module, and the image analysis module determines head images of people walking in each frame and three-dimensional position information of the specified body part by using an image target detection and positioning algorithm. The image analysis module determines whether the head image of the walking person in the image analysis module can be matched with the head image of the walking person determined by the image target detection and positioning algorithm or not by using a matching algorithm, and when the head image of the target 1 in the first frame image is transmitted into the matching algorithm, the head image of the walking person does not exist in the image analysis module, so that the matching algorithm can create a label for the target 1, establish a mapping relation between the head image of the target 1 and the label, and store the head image of the target 1 and the mapping relation between the head image of the target 1 and the label in the image analysis module; and the data processing module records the reference time, the dynamic echo signal and the static echo signal of the target 1 in the first frame image under the corresponding label. The image analysis module determines the head image of the walking person to be matched with the head image of the walking person by using a matching algorithm, and distributes a label corresponding to the head image of the walking person. Thus, the reference time of the target 1 in the images of the second frame, the third frame and the fourth frame, the dynamic echo signal and the static echo signal are all recorded under the corresponding label. And a target 2 in the fifth frame of image enters a monitoring area, the target 1 and the target 2 exist in the fifth frame of image at the same time, and the image analysis module respectively detects head images of two groups of walking people and three-dimensional position information of a specified body part by using an image target detection and positioning algorithm. The image analysis module has the head image of the target 1 and does not have the head image of the target 2, so the matching algorithm allocates the existing label to the head image of the target 1 in the fifth frame and creates a new label for the head image of the target 2. The data processing module stores the reference time, the dynamic echo signal and the static echo signal of the target 1 under the label of the target 1; the reference time, the dynamic echo signal and the static echo signal of the target 2 are stored under the tag of the target 2. When the sixth frame image and the seventh frame image are shot, the target 1 and the target 2 exist in the monitoring area at the same time, the target 1 leaves the monitoring area after the seventh frame image is shot, finally, the seven frame images are used for obtaining the static echo signals of the seven targets 1, the seven static echo signals are used for calculating the physiological data of the target 1, and the physiological data can be heart rate or respiratory rate. After the tenth frame of image is shot, the target 2 leaves the monitoring area, finally, the six frames of images are used for obtaining the static echo signals of the six targets 2, and the physiological data of the target 2 are calculated by using the six static echo curves.

In order to reduce the influence of the attenuation of the electromagnetic wave, in one embodiment, a physiological data monitoring method is provided, which includes the following steps:

step 602, when only one walking person is allowed to pass through the monitoring area at the same time in one way, monitoring physiological data of the walking person by using n monitoring devices arranged in the monitoring area, wherein n is an integer greater than or equal to 2.

In one embodiment of the application, a monitoring device includes an image capture apparatus, a radar, an image analysis module, and a data processing module. A plurality of monitoring devices are arranged in the monitoring area, the intervals among the monitoring devices are the same, and data transmission channels are arranged between the adjacent monitoring devices. The number of the monitoring devices arranged in the monitoring area is related to the length of the monitoring area and the attenuation degree of radar electromagnetic waves in the monitoring device, and the longer the length is, the greater the attenuation degree is, the more the number of the monitoring devices is arranged. If n monitoring devices are arranged in the monitoring area, the monitoring area is uniformly divided into n sections of monitoring subareas, each monitoring device is responsible for monitoring physiological data of people walking in one section of monitoring subarea, and the monitoring devices are placed at the end points of the corresponding monitoring subareas.

Step 604, acquiring images in a monitoring area through a first monitoring device, determining whether a walking person exists in the monitoring area in the images by using an image target detection and positioning algorithm for each acquired frame of image through an image analysis module of the first monitoring device, and if the walking person exists, acquiring a static echo signal of an equivalent static person corresponding to the acquired images in the first monitoring area in an up-and-down state of an appointed body part.

When only one walking person is allowed to pass through the monitoring area in one direction at the same time, as shown in fig. 12, the camera device of the first monitoring device collects images in the monitoring area, the image analysis module in the first monitoring device firstly determines whether the walking person exists in the first monitoring area by using an image target detection and positioning algorithm, if the walking person exists, the three-dimensional position information of the appointed body part of the walking person is determined by using the image target detection and positioning algorithm, and the data processing module of the first monitoring device calculates the distance between the appointed body part and the radar of the first monitoring device according to the three-dimensional position information of the appointed body part and the three-dimensional position information of the radar of the first monitoring device, so that the distance between the appointed body part of the walking person in each frame of images and the radar is obtained in a state that the appointed body part does not fluctuate. An analysis module of the radar of the first monitoring device determines the transmitting direction of the main radar beam of the first monitoring device according to the three-dimensional position information of the designated body part in each frame of image, the mounting position of the radar of the first monitoring device and the mounting posture of the radar of the first monitoring device, and controls the radar of the first monitoring device to transmit electromagnetic waves to the transmitting direction of the main radar beam of the first monitoring device in a preset working mode; the preset working mode comprises a pulse ultra-wideband mode or a frequency modulation continuous wave mode; a data processing module of a first monitoring device acquires dynamic echo signals which are received by a radar of the first monitoring device and aim at the fluctuating state of a specified body part of a walking person in each frame of image; the data processing module of the first monitoring device transforms the distance between the appointed body part of the walking person in each frame of image and the radar in the non-fluctuating state to obtain the reference time of the electromagnetic waves from the radar to the appointed body part and then back to the radar in the non-fluctuating state of the appointed body part of the walking person; and the data processing module of the first monitoring device calculates and obtains the static echo signal of the appointed body part fluctuation state of the equivalent static person corresponding to each frame of image according to the reference time and the dynamic echo signal corresponding to each frame of image. The above technical processes are detailed in the above embodiments and will not be described herein.

Step 606, when the walking person walks out of the first monitoring subarea, the first monitoring device sends a static echo signal to a second monitoring device; and the second monitoring device obtains a static echo signal of the appointed body part of the equivalent static person corresponding to the acquired image in the second monitoring subarea in the fluctuating state.

In an embodiment of the application, since the monitoring devices are placed at the end points of the monitoring subareas, when the image analysis module in the first monitoring device receives a frame of image which does not contain a walking person after receiving a plurality of frames of images which contain the walking person located in the first monitoring subarea, it indicates that the walking person has walked out of the first monitoring subarea. The first monitoring device stops monitoring the physiological data of the walking person and sends the static echo signal in the data processing module of the first monitoring device to the second monitoring device. And the second monitoring device receives the information transmitted by the first monitoring device and starts monitoring the physiological data.

At this time, the image analysis module of the second monitoring device does not need to judge whether the walking person enters the second monitoring subarea, the data transmitted by the first monitoring device can be used as a signal for the second monitoring device to start physiological data monitoring, and the second monitoring device obtains the static echo signal of the equivalent static person in the fluctuation state of the specified body part corresponding to the acquired image in the second monitoring subarea by using the camera device of the second monitoring device, the radar of the second monitoring device, the image analysis module of the second monitoring device and the data processing module of the second monitoring device. When the image analysis module in the second monitoring device receives a plurality of frames of images containing the walking people in the second monitoring subarea, and receives a frame of image not containing the walking people, the walking people are shown to have walked out of the second monitoring subarea. And the second monitoring device stops physiological data monitoring and sends the static echo signal corresponding to the first monitoring device and the static echo signal corresponding to the second monitoring device to a third monitoring device.

Step 608, when the walking person walks out of the nth-1 monitoring subarea, the nth-1 monitoring device sends the static echo signals corresponding to the first monitoring device to the nth-1 monitoring device; the nth monitoring device obtains static echo signals of the equivalent static human body part in the fluctuating state corresponding to the collected image in the nth subarea, and the data processing module of the nth monitoring device extracts the physiological data of the walking human body from the static echo signals corresponding to the first to nth monitoring devices.

In an embodiment of the application, when a walking person walks out of an n-1 th monitoring subarea, the n-1 th monitoring device sends static echo signals corresponding to the first to the n-1 th monitoring devices to a last monitoring device, namely an nth monitoring device, and after the nth monitoring device calculates the static echo signals, a data processing module of the nth monitoring device calculates the physiological data of the walking person according to the static echo signals corresponding to the first to the nth monitoring devices.

In this embodiment, when only one walking person is allowed to pass through in the same time in the monitoring area, compared with a scheme that one radar is used for transmitting electromagnetic waves to the walking person in the monitoring area and receiving an echo, a plurality of monitoring devices are arranged in the monitoring area to monitor physiological data of the walking person, the distance between the radar in each monitoring device and the walking person is closer, the received echo signal is more accurate, and the physiological data calculated by the echo signal is more accurate.

In one embodiment, when a monitoring area unidirectionally allows a plurality of walking persons to pass through at the same time, the technical solution is different from the technical solution when only one walking person is unidirectionally allowed to pass through at the same time in the monitoring area, a tag needs to be created for the walking person according to the head image of the walking person, and the head image and the tag of the walking person are stored, and the rest technical processes are detailed in the above embodiments. An image analysis module in the first monitoring device firstly determines whether a walking person exists in a first monitoring subarea by using an image target detection and positioning algorithm and a matching algorithm, if so, judges whether the head image of the walking person can be matched with the existing image in the image analysis module, and if not, creates a label for the walking person and stores the head image of the walking person. When the walking person walks out of the first monitoring subarea, the first monitoring device of the appointed body part of the walking person stops monitoring the physiological data of the walking person, and the head image of the walking person, the labels corresponding to the head image of the walking person and the static echo signals under each label in the data processing module, which are stored in the image analysis module in the first monitoring device, are sent to the second monitoring device. The second monitoring device receives the information transmitted by the first monitoring device and starts physiological data monitoring, an image analysis module of the second monitoring device stores the head image of the walking person and a corresponding label transmitted by the first monitoring device, because the person walking through the second monitoring device passes through the first monitoring device first, the second monitoring device does not need to create a label, and the data transmitted by the first monitoring device is used for monitoring continuously, when the walking person walks out of the second monitoring subarea, the second monitoring device stops the physiological data monitoring, and sending the head image of the walking person in the image analysis module, the corresponding label, the static echo signal obtained by the first monitoring device and the static echo signal obtained by the second monitoring device under each label in the data processing module to the third monitoring device, and so on. Therefore, except for the first monitoring device, the subsequent monitoring devices do not need to judge whether the label needs to be created or not. And calculating by a data processing module in the last monitoring device according to the static echo signals of all the monitoring devices to obtain the physiological data of the person walking under each label.

In this embodiment, when a plurality of walking persons are allowed to pass through the monitoring area in one direction at the same time, compared with a scheme of using one radar to transmit electromagnetic waves to the walking persons in the monitoring area and receive echoes, after a plurality of monitoring devices are arranged, the distance between the radar in each monitoring device and the walking persons is shorter, the received echo signals are more accurate, and the physiological data calculated by the echo signals are more accurate; then, a plurality of walking persons can be monitored at the same time, thereby improving the monitoring efficiency of physiological data.

It should be understood that although the steps in the flowcharts of fig. 4 and 10 are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 4 and 10 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least some of the other steps or stages.

In one embodiment, as shown in fig. 13, there is provided a physiological data monitoring device 1300 comprising: image acquisition module 1302, position determination module 1304, distance determination module 1306, direction determination module 1308, signal acquisition module 1310, transformation module 1312, motion compensation module 1314, and data extraction module 1316.

The image acquiring module 1302 is configured to acquire an image in a monitoring area by using a camera.

And a position determining module 1304, configured to determine, for each acquired frame of image, whether a walking person is present in the monitoring area in the image by using an image target detection and positioning algorithm when only one walking person is allowed to pass through the monitoring area at the same time, and if a walking person is present in the monitoring area, determine three-dimensional position information of a specified body part of the walking person by using the image target detection and positioning algorithm.

And the distance determining module 1306 is configured to calculate a distance between the designated body part and the radar according to the three-dimensional position information of the designated body part and the three-dimensional position information of the radar, and obtain a distance between the designated body part of the person walking in each frame of image and the radar in a non-fluctuating state.

A direction determining module 1308, configured to determine a transmitting direction of the radar main beam according to the three-dimensional position information of the designated body part in each frame of image, the installation position of the radar, and the installation posture of the radar, and control the radar to transmit an electromagnetic wave to the transmitting direction of the radar main beam in a preset working mode; the preset working mode comprises a pulse ultra-wideband mode or a frequency modulation continuous wave mode.

A signal obtaining module 1310, configured to obtain dynamic echo signals received by the radar in a fluctuating state of a specified body part of a walking person in each frame of image.

And a transforming module 1312 for transforming the distance between the specified body part of the walking person in each frame of image and the radar in the no-fluctuation state to obtain the reference time of the electromagnetic wave from the radar to the specified body part and back to the radar in the no-fluctuation state of the specified body part of the walking person.

And a motion compensation module 1314, configured to calculate a static echo signal in an undulating state of the equivalent static human designated body part corresponding to each frame of image according to the reference time corresponding to each frame of image and the dynamic echo signal.

The data extraction module 1316 is configured to extract physiological data of the walking person from a plurality of static echo signals corresponding to the plurality of frames of images.

In an optional embodiment of the present application, the motion compensation module 1314 is specifically configured to acquire a dynamic echo signal of an ith frame of image in an image corresponding to a fluctuating state of a specified body part when an operation mode of the radar is impulse ultra-wideband

For the dynamic echo signal

Where j is an imaginary unit and ω is the echo signal

Angular frequency, t, after Fourier transformation_iAnd the reference time of the electromagnetic wave from the radar to the specified body part and then returning to the radar in the state that the specified body part of the walking person corresponding to the image i is not fluctuated.

In an optional embodiment of the present application, the motion compensation module 1314 is specifically configured to, when the radar operating mode is frequency modulated continuous wave, acquire a dynamic echo signal of an ith frame of image in the image corresponding to the fluctuation state of the specified body part

For the dynamic echo signal

Where j is an imaginary unit and ω is the echo signal

Angular frequency, t, after two Fourier transforms_iAnd the reference time of the electromagnetic wave from the radar to the specified body part and then returning to the radar in the state that the specified body part of the walking person corresponding to the image i is not fluctuated.

In one embodiment, the radar is a phased array radar, as shown in fig. 14, which shows a block diagram of another physiological data monitoring device 1400 provided by an embodiment of the present application, the physiological data monitoring device 1400 includes an image acquisition module 1302 and a data extraction module 1316 of the physiological data monitoring device 1300, and the optional physiological data monitoring device 1400 further includes a detection location module 1402, a tag creation module 1404, and a recording module 1406.

The detection and positioning module 1402 is configured to determine, for each acquired frame image, whether a walking person exists in the monitoring area by using an image target detection and positioning algorithm when a plurality of walking persons are allowed to pass through the monitoring area at the same time, and if a walking person exists in the monitoring area, determine, by using the image target detection and positioning algorithm, a head image of each walking person and three-dimensional position information of a designated body part of the walking person in each frame image.

The label creating module 1404 is configured to match the head image of each walking person with the head images of the existing walking persons by using a matching algorithm, and create a label for each walking person who enters the monitoring area for the first time according to a matching result.

The recording module 1406 is configured to record the static echo signal of each walking person under the corresponding tag.

The embodiment of the application provides another physiological data monitoring device, which comprises a monitoring module.

The monitoring module is used for monitoring physiological data of a walking person by using n monitoring devices arranged in a monitoring area when the walking person is allowed to pass through the monitoring area in one direction at the same time, wherein n is an integer greater than or equal to 2.

The monitoring module is specifically used for acquiring images in a monitoring area through a first monitoring device, aiming at each acquired frame of image, an image analysis module of the first monitoring device determines whether a walking person exists in the monitoring area in the image by using an image target detection and positioning algorithm, and if the walking person exists, a static echo signal in an up-and-down state of an appointed body part of an equivalent static person corresponding to the acquired images in the first monitoring area is acquired; when the walking person walks out of the first monitoring subarea, the first monitoring device sends a static echo signal to the second monitoring device; the second monitoring device obtains a static echo signal of the appointed body part of the equivalent static person corresponding to the collected image in the second monitoring subarea in the fluctuating state; when the walking person walks out of the second monitoring subarea, the second monitoring device sends the static echo signal corresponding to the first monitoring device and the static echo signal corresponding to the second monitoring device to a third monitoring device; until the walking person walks out of the nth-1 monitoring subarea, the nth-1 monitoring device sends the static echo signals corresponding to the first monitoring device to the nth-1 monitoring device; the nth monitoring device obtains static echo signals of the equivalent static human body part in the fluctuating state corresponding to the collected image in the nth subarea, and the data processing module of the nth monitoring device extracts the physiological data of the walking human body from the static echo signals corresponding to the first to nth monitoring devices.

As shown in fig. 15, which illustrates a block diagram of another physiological data monitoring device 1500 provided in the embodiment of the present application, the physiological data monitoring device 1500 further includes an optional result analysis module 1502 in addition to the various modules included in the physiological data monitoring device 1300.

The result analysis module 1502 is configured to input the physiological data of the walking person into an artificial neural network, so as to obtain the health condition of the person.

The data extraction module 1316 obtains a sequence of physiological data, which may be a sequence of a respiration rate and a heart rate in an optional embodiment of the present application, and inputs the sequence of the respiration rate and the heart rate into an artificial neural network model, and the artificial neural network model outputs a health condition of a human body.

In this embodiment, a result analysis module 1502 is added to input the physiological data into the artificial neural network model, and the artificial neural network model outputs the health condition of the human body. The device with the result analysis module can analyze the physiological data, and the health condition of the human body can be obtained in the walking process of the human body by arranging the device on the channel, so that the accurate real-time health condition of the walking person can be obtained.

In a specific embodiment, as shown in fig. 16, the image obtaining module 1302 of the physiological data monitoring apparatus performs video capture, the captured image is immediately transmitted to the position determining module 1304, in which it is determined whether there is a walking person in the monitoring area based on the image target detecting and positioning algorithm of the deep learning model, if there is a walking person, the deep learning model outputs three-dimensional position information of a designated body part of the walking person, the three-dimensional position information of the designated body part is transmitted to the multiple information displaying module, the distance determining module 1306 and the direction determining module 1308, and after the distance determining module 1306 receives the three-dimensional position information of the designated body part, the distance between the designated body part and the radar is calculated by using the three-dimensional position information, and the distance is transmitted to the transforming module 1312. The transform module transforms the distance to a reference time and passes the reference time to the motion compensation module 1314. After the direction determining module 1308 receives the three-dimensional position information of the designated body part, the three-dimensional position information of the designated body part is used to determine the transmitting direction of the main beam of the radar, and the phased array radar or the mechanical rotation radar transmits electromagnetic waves to the determined direction and transmits the received dynamic echo signal to the motion compensation module 1314. The motion compensation module 1314 performs motion compensation on the received dynamic echo signal by using the reference time to obtain a static echo signal, and when the measured physiological data is respiration and heart rate, the data extraction module 1316 extracts the respiration rate and the heart rate of the walking person from the static echo signal by using a conventional respiration rate and heart rate extraction algorithm, and transmits the respiration rate and the heart rate to the multiple information display module for display. The data extraction module 1316 also transmits the respiration rate and heart rate information to the result analysis module 1502, and inputs the sequence of the physiological data into the artificial neural network model, which outputs the health condition of the human body, and the staff in the monitoring area handles the walking people with different health conditions in a classified manner.

For specific limitations of the physiological data monitoring device, reference may be made to the above limitations of the physiological data monitoring method, which are not described herein again. The modules in the physiological data monitoring device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 17. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a physiological data monitoring method.

Those skilled in the art will appreciate that the architecture shown in fig. 17 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.

In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method of physiological data monitoring, the method comprising:

collecting images in a monitoring area by utilizing camera equipment;

when only one walking person is allowed to pass through the monitoring area in one way at the same time, determining whether the walking person exists in the monitoring area in the image or not by utilizing an image target detection and positioning algorithm aiming at each acquired frame of image, and if the walking person exists in the monitoring area, determining the three-dimensional position information of the appointed body part of the walking person by utilizing the image target detection and positioning algorithm;

calculating the distance between the designated body part and the radar according to the three-dimensional position information of the designated body part and the three-dimensional position information of the radar to obtain the distance between the designated body part of the walking person in each frame of image and the radar in a non-fluctuating state;

determining the transmitting direction of a radar main beam according to the three-dimensional position information of the designated body part in each frame of image, the mounting position of the radar and the mounting posture of the radar, and controlling the radar to transmit electromagnetic waves to the transmitting direction of the radar main beam in a preset working mode; the preset working mode comprises a pulse ultra-wideband mode or a frequency modulation continuous wave mode;

acquiring dynamic echo signals received by the radar under the fluctuation state of the appointed body part of a walking person in each frame of image;

converting the distance between the appointed body part of the walking person in each frame of image and the radar in a non-fluctuating state to obtain the reference time of the electromagnetic waves from the radar to the appointed body part and then returning to the radar in the non-fluctuating state of the appointed body part of the walking person;

calculating to obtain a static echo signal of an equivalent static human body part corresponding to each frame of image in an up-and-down state according to the reference time and the dynamic echo signal corresponding to each frame of image;

and extracting physiological data of the walking person from a plurality of static echo signals corresponding to the multi-frame images.

2. The method of claim 1, wherein calculating the static echo signal under the fluctuating state of the designated body part of the equivalent static person corresponding to each frame of image according to the reference time and the dynamic echo signal corresponding to each frame of image comprises:

For the dynamic echoSignal

Fourier transform and exp (j ω t) multiplication_i) Then, Fourier inversion is carried out to obtain a static echo signal of the equivalent static human corresponding to the ith frame image under the fluctuating state of the appointed body part

3. The method of claim 1, wherein calculating the static echo signal under the fluctuating state of the designated body part of the equivalent static person corresponding to each frame of image according to the reference time and the dynamic echo signal corresponding to each frame of image comprises:

For the dynamic echo signal

4. The method of claim 1, wherein the radar is a phased array radar; the method further comprises the following steps:

when a plurality of walking persons are allowed to pass through the monitoring area in one way at the same time, determining whether the walking persons exist in the monitoring area in each acquired frame image by using an image target detection and positioning algorithm, and if the walking persons exist in the monitoring area, determining the head image of each walking person and the three-dimensional position information of the appointed body part of the walking person in each frame image by using the image target detection and positioning algorithm;

matching the head image of each walking person with the head images of the walking persons by using a matching algorithm, and creating a label for each walking person entering a monitoring area for the first time according to a matching result;

recording the static echo signal of each walking person under the corresponding tag.

5. A method of physiological data monitoring, the method comprising:

monitoring physiological data of a walking person by using n monitoring devices arranged in the monitoring area when only one walking person is allowed to pass through the monitoring area at the same time, wherein n is an integer greater than or equal to 2;

acquiring images in a monitoring area through a first monitoring device, determining whether a walking person exists in the monitoring area in the images by using an image target detection and positioning algorithm aiming at each acquired frame of image through an image analysis module of the first monitoring device, and if the walking person exists, acquiring a static echo signal of an equivalent static person corresponding to the acquired images in the first monitoring area under the fluctuation state of an appointed body part;

when the walking person walks out of the first monitoring subarea, the first monitoring device sends a static echo signal to the second monitoring device; the second monitoring device obtains a static echo signal of the appointed body part of the equivalent static person corresponding to the collected image in the second monitoring subarea in the fluctuating state;

when the walking person walks out of the second monitoring subarea, the second monitoring device sends the static echo signal corresponding to the first monitoring device and the static echo signal corresponding to the second monitoring device to a third monitoring device;

until the walking person walks out of the nth-1 monitoring subarea, the nth-1 monitoring device sends the static echo signals corresponding to the first monitoring device to the nth-1 monitoring device; the nth monitoring device obtains static echo signals of the equivalent static human body part in the fluctuating state corresponding to the collected image in the nth subarea, and the data processing module of the nth monitoring device extracts the physiological data of the walking human body from the static echo signals corresponding to the first to nth monitoring devices.

6. The method of claim 5, wherein the monitoring device comprises a camera, a radar, an image analysis module, and a data processing module; wherein the method further comprises:

acquiring an image in a monitoring area through the camera equipment;

when only one walking person is allowed to pass through the monitoring area in one direction at the same moment, aiming at each frame of image, determining the three-dimensional position information of the appointed body part of the walking person by using an image target detection and positioning algorithm through the image analysis module;

calculating the distance between the designated body and the radar through the data processing module according to the three-dimensional position information of the designated body part and the three-dimensional position information of the radar to obtain the distance between the designated body part of the walking person in each frame of image and the radar in a non-fluctuating state;

determining the emission direction of the radar main beam according to the three-dimensional position information of the designated body part in each frame of image, the installation position of the radar and the installation posture of the radar by the analysis module of the radar, and controlling the radar to emit electromagnetic waves to the emission direction of the radar main beam in a preset working mode; the preset working mode comprises a pulse ultra-wideband mode or a frequency modulation continuous wave mode;

acquiring dynamic echo signals received by the radar under the fluctuation state of the appointed body part of a walking person in each frame of image through the data processing module;

converting the distance between the designated body part of the walking person in each frame of image and the radar in a non-fluctuating state through the data processing module to obtain the reference time of the electromagnetic waves from the radar to the designated body part and then returning to the radar in the non-fluctuating state of the designated body part of the walking person;

and calculating to obtain a static echo signal of the equivalent static human body part corresponding to each frame of image in the fluctuating state according to the reference time corresponding to each frame of image and the dynamic echo signal by the data processing module.

7. A physiological data monitoring device, the device comprising:

the position determining module is used for determining whether a walking person exists in the monitoring area in the image by using an image target detection and positioning algorithm aiming at each acquired frame of image when only one walking person is allowed to pass through the monitoring area at the same time, and if the walking person exists in the monitoring area, determining the three-dimensional position information of the appointed body part of the walking person by using the image target detection and positioning algorithm;

the motion compensation module is used for calculating and obtaining a static echo signal corresponding to each frame of image under the fluctuation state of the equivalent static human designated body part according to the reference time and the dynamic echo signal corresponding to each frame of image;

8. The apparatus of claim 7, further comprising:

and the result analysis module is used for inputting the physiological data of the walking person into an artificial neural network to obtain the health condition of the person.

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 6.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.