CN117434509A

CN117434509A - Method, device, terminal equipment and storage medium for acquiring vital sign data

Info

Publication number: CN117434509A
Application number: CN202311379097.7A
Authority: CN
Inventors: 程思国
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2023-10-23
Filing date: 2023-10-23
Publication date: 2024-01-23

Abstract

The embodiment of the application discloses a method, a device, a terminal device and a storage medium for acquiring vital sign data, wherein the terminal device comprises a radar system, so that raw data can be acquired directly through the radar system, direct contact with a user is not needed, the raw data is processed through a preset network model, the vital sign data is acquired, and the convenience for acquiring the vital sign data of the user is greatly improved. The method of the embodiment of the application is applied to terminal equipment, wherein the terminal equipment comprises a radar system, and the method can comprise the following steps: under the condition that the radar system is in an on state, acquiring first original data through the radar system; under the condition that the radar system is in an on state, acquiring first original data through the radar system; inputting the first original data into a first preset network model, and outputting vital sign data, wherein the first preset network model is an algorithm model obtained by model training according to a first historical original data set and a historical vital sign data set.

Description

Method, device, terminal equipment and storage medium for acquiring vital sign data

Technical Field

The present invention relates to the field of data acquisition, and in particular, to a method, an apparatus, a terminal device, and a storage medium for acquiring vital sign data.

Background

The detection of vital sign data is basically required to be contacted by the prior art, so that the prior art is generally required to be made into a form of a bracelet/watch, which can prevent the use of related functions from being popular to some extent, and the prior art is inconvenient to carry.

Disclosure of Invention

The embodiment of the application provides a method, a device, a terminal device and a storage medium for acquiring vital sign data, wherein the terminal device comprises a radar system, so that raw data can be acquired directly through the radar system, direct contact with a user is not needed, the raw data is processed through a preset network model, the vital sign data is acquired, and convenience for acquiring the vital sign data of the user is greatly improved.

A first aspect of the present application provides a method for acquiring vital sign data, the method being applied to a terminal device, the terminal device comprising a radar system, the method may include: under the condition that the radar system is in an on state, acquiring first original data through the radar system; inputting the first original data into a first preset network model, and outputting vital sign data, wherein the first preset network model is an algorithm model obtained by model training according to a first historical original data set and a historical vital sign data set.

A second aspect of the present application provides an apparatus for acquiring vital sign data, the apparatus being applied to a terminal device, the terminal device including a radar system, the apparatus may include:

the acquisition module is used for acquiring first original data through the radar system under the condition that the radar system is in an on state;

the processing module is used for inputting the first original data into a first preset network model and outputting vital sign data, and the first preset network model is an algorithm model obtained by model training according to a first historical original data set and a historical vital sign data set.

A third aspect of the present application provides a terminal device, which may include:

a memory storing executable program code;

a processor coupled to the memory;

the processor is configured to correspondingly perform the method according to the first aspect of the present application.

Yet another aspect of an embodiment of the present application provides a computer-readable storage medium comprising instructions which, when run on a processor, cause the processor to perform the method described in the first aspect of the present application.

A further aspect of the embodiments of the present application discloses a computer program product which, when run on a computer, causes the computer to perform the method of the first aspect of the present application.

A further aspect of the embodiments of the present application discloses an application publishing platform for publishing a computer program product, wherein the computer program product, when run on a computer, causes the computer to perform the method according to the first aspect of the present application.

From the above technical solutions, the embodiments of the present application have the following advantages:

in the embodiment of the application, the method for acquiring vital sign data is applied to a terminal device, wherein the terminal device comprises a radar system, and the method comprises the following steps: under the condition that the radar system is in an on state, acquiring first original data through the radar system; inputting the first original data into a first preset network model, and outputting vital sign data, wherein the first preset network model is an algorithm model obtained by model training according to a first historical original data set and a historical vital sign data set. Because the terminal equipment comprises the radar system, the original data can be directly acquired through the radar system, and the original data is processed by the preset network model without direct contact with a user, so that the vital sign data is obtained, and the convenience for acquiring the vital sign data of the user is greatly improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly introduce the drawings that are needed in the embodiments and the description of the prior art, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings.

FIG. 1 is a schematic diagram of a chip integrated with a radar system according to an embodiment of the present application;

FIG. 2 is a schematic diagram of an embodiment of a method for acquiring vital sign data according to an embodiment of the present application;

fig. 3A is a schematic view of a scenario applied in an embodiment of the present application;

FIG. 3B is a schematic diagram of acquiring first raw data of a vital sign data detection mode by a radar system according to an embodiment of the present application;

FIG. 3C is a schematic diagram illustrating the overall network operation in an embodiment of the present application;

FIG. 4 is a schematic diagram of another embodiment of a method for acquiring vital sign data according to an embodiment of the present application;

FIG. 5A is a flowchart of a method for acquiring vital sign data according to an embodiment of the present application;

FIG. 5B is a diagram showing vital sign data according to an embodiment of the present application;

fig. 5C is a schematic flow chart of unlocking by combining face recognition with vital sign data in the embodiment of the present application;

FIG. 5D is a schematic flow chart of heartbeat/respiration and gesture detection in an embodiment of the present application;

FIG. 5E is a schematic diagram of acquiring second raw data of a gesture detection mode by a radar system according to an embodiment of the present application;

FIG. 6A is a schematic diagram of an embodiment of an apparatus for acquiring vital sign data according to an embodiment of the present application;

fig. 6B is a schematic diagram of an embodiment of a terminal device in an embodiment of the present application;

fig. 7 is a schematic diagram of another embodiment of a terminal device in an embodiment of the present application.

Detailed Description

In order for those skilled in the art to better understand the present application, the following description will describe embodiments of the present application with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. Based on the examples in this application, all shall fall within the scope of protection of this application.

The existing human health state data detection technology basically needs close contact, and the existing two methods mainly comprise the following steps:

1. PPG (Photo Plethysmo Graphy) photoplethysmography principle

The method is based on the principle that the intelligent bracelet uses most at present, and simply applies the light reflection principle to convert the pulse change according to the light transmittance in blood into corresponding electric signals, and then applies a preset algorithm to convert the heart rate data according to the known electric signals. The skin surface is irradiated by a beam with fixed wavelength, the beam returns to the photoelectric receiver by reflection, transmission and the like, when the heart contracts, the blood volume in the body is the most, the light absorption amount is the most, and the detected light intensity is the least; in the same way, when the heart is relaxed, the light absorption is minimum and the light intensity is maximum, so that the periodical change of the light intensity along with the pulse is realized; green light is generally used because blood is mostly red, and the absorption rate of green light is greatest, so that heart rate monitoring can be more accurate.

2. Electrocardiographic signal measurements, like Electrocardiograph (ECG)

The heart rate detection method uses the human body potential difference to detect the heart rate, is similar to the electrocardiogram principle of hospitals, body fluid and tissues around the heart can be conductive, and each time of heart beat, the myocardial cells work to enable the surface of the human body to generate tiny electrode changes, and after the electrode changes are captured by equipment, the heart rate beat frequency can be restored through professional algorithm calculation.

3. Heart rate refers to the number of beats per minute in a normal person's resting state, also called resting heart rate, generally 60-100 beats/minute, and may vary from one another due to age, gender, or other physiological factors. Generally, the older the age, the faster the heart rate, the slower the elderly heart rate than the young, and the faster the female heart rate than the same age men, all of which are normal physiological phenomena. In a resting state, the normal heart rate of an adult is 60 to 100 times/min, and the ideal heart rate is 55 to 70 times/min (the heart rate of an athlete is slower than that of a common adult, and is generally about 50 times/min).

4. Respiration refers to the process of gas exchange between the body and the external environment. The respiratory process of a person includes three interrelated links: external respiration, including pulmonary ventilation and pulmonary ventilation; transport of gas in blood; internal respiration refers to the exchange of gases between tissue cells and blood. The normal adult is at rest breathing for a time of 6.4 seconds, with a volume of gas inhaled and exhaled each time of approximately 500 ml, known as tidal volume. When a person inhales forcefully until the person can not inhale any more; then exhaling with force until the time when the exhalation is no longer possible, at which time the volume of exhaled gas is called the lung capacity.

The technical scheme of the application is further described below by way of examples. The terminal device in the embodiment of the application comprises a radar system, wherein the radar system can comprise the following modules: a Radio Frequency (RF) front end, an analog Baseband (ABB), an analog-to-digital converter (AnalogTo Digital Converter, ADC), a Phase-locked loop (PLL), a memory (e.g., first-in first-out (First Input First Output, FIFO)), a serial peripheral interface (Serial Peripheral Interface, SPI), and an antenna.

Fig. 1 is a schematic diagram of a chip integrated with a radar system according to an embodiment of the present application. A frequency modulated continuous wave (Frequency Modulated Continuous Wave, FMCW) signal is transmitted over a transmit channel (TX) and an echo signal (RX) channel(s) from a target object is received at each receiver. Each receiver path includes a baseband filter, a video graphics array (Video Graphics Array, VGA) and an ADC. The digitized output is stored in a first-in first-out (First Input First Output, FIFO) memory. The data is transmitted to an external host, a microcontroller unit (Microcontroller Unit, MCU) or an Application Processor (AP), which processes the radar signals.

As shown in fig. 2, an embodiment of a method for acquiring vital sign data in an embodiment of the present application is shown, where the embodiment of the method is applied to a terminal device, where the terminal device includes a radar system, and the embodiment of the method includes:

201. and under the condition that the radar system is in an on state, acquiring first original data through the radar system.

Since the radar system is integrated on the chip of the terminal device, the terminal device also has the function of the radar system. The user can open the radar system switch, the terminal equipment responds to the operation of opening the radar system switch by the user, and the radar system switch is opened, so that the radar system is in an on state, and then the original data are collected through the radar system.

Optionally, the wavelength of the radar system is millimeter wave; the radar system has the function of integrating receiving and transmitting.

Fig. 3A is a schematic view of a scenario applied in the embodiment of the present application. Because the radar system is transceiver-integrated, the miniature antenna is integrated on the radar system, the antenna area is small (for example, 5 x 6.5 mm), and the power consumption is low. It can be calibratedDetermining perception of movement in a space, e.g. 1m ³ Inside. Therefore, the low-power radar system with the duplex transceiver (with the transceiver antenna) is integrated on the terminal equipment (such as a mobile phone), and the mobile phone is popular as a communication tool for the current user, and does not need to be in close physical contact with a human body like a watch/bracelet, so that the convenience of detecting vital sign data can be obviously improved.

Alternatively, the radar system switch may be a physical switch or a virtual icon switch.

Optionally, the responding to the operation of opening the radar system switch by the user opens the radar system, so that the radar system is in an open state, and may include: and responding to the operation of opening the virtual icon switch by a user, and opening the radar system to enable the radar system to be in an on state.

Optionally, the operation of opening the virtual icon switch includes an operation of sliding (e.g., sliding from left to right, sliding from top to bottom, etc.) the virtual icon switch, an operation of clicking (or double-clicking) the virtual icon switch, or an operation of inputting a password, etc.

Alternatively, the raw data is binary data sampled consecutively in units of time, and the binary data may be stored in a memory (e.g., FIFO memory) in a filling manner. When the data is read from the memory, 16-ary data (e.g., 87e7afe3cba34df7fb8 …) is obtained. The 16-system data are simply preprocessed to obtain a waveform diagram, and the waveform diagram displays the telecontrol amplitude (i.e. distance) of the detected object at each moment on a time axis, so that the telecontrol period (i.e. frequency) of the object can be calculated. Fig. 3B is a schematic diagram of acquiring first raw data of a vital sign data detection mode by the radar system according to an embodiment of the present application.

202. And inputting the first original data into a first preset network model, and outputting vital sign data.

The first preset network model is an algorithm model obtained by model training according to a first historical original data set and a historical vital sign data set.

Optionally, the method may further include: acquiring the first historical original data set and the historical vital sign data set; and performing model training on the historical original data set and the historical vital sign data set to obtain the first preset network model.

Optionally, the first preset network model is a three-layer or more than three-layer sensing network, and includes an input layer, an implicit layer and an output layer.

Optionally, the performing model training on the first historical original data set and the historical vital sign data set to obtain the first preset network model may include: the first historical original data set is transmitted from an input layer to an output layer through an implicit layer, so that an actual vital sign data set is obtained; correcting a connection weight from an output layer to the hidden layer according to an actual vital sign data set and the historical vital sign data set and an error square minimum rule; in this way, when the actual vital sign data set is the historical vital sign data set, or when the error between the actual vital sign data set and the historical vital sign data set is smaller than a first preset threshold value, or when the preset learning times are reached, determining the connection weight of each layer; and obtaining the first preset network model according to the first historical original data set and the connection weight of each layer.

Optionally, the method may further include: acquiring an updated first original data set and updated vital sign data; and updating the first preset network model according to the updated first original data and the updated vital sign data to obtain an updated first preset network model.

It is understood that the first preset network model is an artificial intelligence AI-based neural network model. Optionally, the preset network model may include, but is not limited to: BP neural network, normal convolutional neural network, separate convolutional neural network, transposed convolutional neural network, full-connection layer neural network, etc.

Fig. 3C is a schematic diagram of the whole network operation process in the embodiment of the present application. In fig. 3C, the predetermined network model is a perceived network having three or more layers. Among these, a typical multi-layer perceptive network is three layers, namely: input layer, hidden layer (also called middle layer) and output layer. The neurons of the adjacent layers of the three-layer network are fully connected, namely, each neuron of the lower layer and each neuron of the upper layer are fully connected, and each neuron of each layer is not connected, and the connection strength forms a weight matrix W of the network.

It can be understood that, the BP neural network first sets a desired output value for each input learning and memorizing mode, and then, in the embodiment of the present application, each system state corresponds to one vital sign data. The learning and memorizing mode is then input to the network and propagated from the input layer to the output layer via the intermediate layer. The difference between the actual output value and the desired output value is the error. According to the rule of minimum error square, the connection weight is corrected layer by layer from the output layer to the middle layer, and the process is called error back propagation. With alternating iterations of the "mode forward" and "error reverse" processes. The actual output values of the BP neural network gradually approach to the corresponding expected output values, and the accuracy of the response of the BP neural network to the input learning and memory mode also continuously rises. Through the learning process, the connection weight between the lower layers is determined.

It should be noted that, after the error of the BP neural network is reduced to a certain extent through multiple learning, the further reduction is very slow, if the error is within the range accepted by the user, the learning can be stopped, and at this time, the obtained model can be regarded as the preset neural network model. The range accepted by the user can be considered herein to be the first preset threshold. Theoretically, the smaller the first preset threshold, the higher the accuracy of the preset neural network model. The value of the first preset threshold may be determined according to an empirical value, or may be determined according to an actual requirement of a user, which is not specifically limited herein.

The preset learning times can be an experience value or can be determined according to the actual requirement of the user, and generally, after the preset learning times are reached, the generated error is within the range accepted by the user. In theory, the more the learning times, the smaller the generated error, but in practical application, the unlimited learning is impossible, so that a preset learning time can be set, and after the preset learning time is reached, the obtained model can be regarded as a preset neural network model.

In the embodiment of the application, the method for acquiring vital sign data is applied to a terminal device, wherein the terminal device comprises a radar system, and the method comprises the following steps: under the condition that the radar system is in an on state, acquiring first original data through the radar system; inputting the first original data into a first preset network model, and outputting vital sign data, wherein the first preset network model is an algorithm model obtained by model training according to a first historical original data set and a historical vital sign data set. Because the terminal equipment comprises the radar system, the original data can be directly acquired through the radar system, and the original data is processed by the preset network model without direct contact with a user, so that the vital sign data is obtained, and the convenience for acquiring the vital sign data of the user is greatly improved. The low-power consumption millimeter wave radar system is integrated on the terminal equipment, so that physical contact is not needed, and at least one of heartbeat and respiration of a user can be conveniently detected in real time only by approaching the radar system, and the psychological change of emotion of the user can be recorded.

As shown in fig. 4, another embodiment of a method for acquiring vital sign data in an embodiment of the present application is shown, where the embodiment of the method is applied to a terminal device, where the terminal device includes a radar system, and the embodiment of the method includes:

401. and under the condition that the radar system is in an on state, acquiring first original data through the radar system.

402. And inputting the first original data into a first preset network model, and outputting vital sign data.

It should be noted that 401 and 402 in the embodiment of the present application are similar to steps 201 and 202 in the embodiment shown in fig. 2, and are not described herein.

403. And displaying the vital sign data on a display interface.

Optionally, the vital sign data includes: at least one of heart rate and respiration. It is understood that the heart rate may also be referred to as a heartbeat.

Optionally, the displaying the vital sign data on the display interface may include:

displaying the vital sign data on a display interface in the form of a popup window; or alternatively, the first and second heat exchangers may be,

and displaying the vital sign data on a target display interface of the split screen, wherein an opened application program interface is not displayed on the target display interface.

Optionally, the method may further include: and saving the vital sign data. Can be used for users to inquire the vital sign data later; and forming a life body characteristic health report of the user according to the life body characteristic data of each period, and evaluating the health condition of the user.

404. And judging the current health condition of the user according to the vital sign data.

It will be appreciated that the vital sign data acquired is periodic. Alternatively, the period may be the same as or different from the period in which the raw data was acquired. In general, the period of vital sign data is minutes or seconds. The vital sign data for each cycle may be a mean value of the vital sign data for that cycle.

Optionally, the vital sign data includes a heart rate, and the terminal device determines the current health condition of the user according to the vital sign data, and may include: judging that the current heart rate health condition of the user is in a health state under the condition that the heart rate is in a preset heart rate range; and under the condition that the heart rate is out of the preset heart rate range, judging that the current heart rate health condition of the user is in a non-health state. According to the method and the device for judging the heart rate health condition of the user, the current heart rate health condition of the user can be judged according to the vital sign data acquired in the current period.

Optionally, the method may further include: and in the first preset duration, judging that the current heart rate health state of the user is in an unhealthy state when the heart rate of each period is outside a preset heart rate range or when the number of the periods with the heart rate outside the preset heart rate range exceeds the preset period number. It will be appreciated that the first preset duration of the present application is generally understood as data having a sufficient duration, otherwise not having high reliability, for example, if the preset duration is 1s, it is possible that the user sits for a long time, stands up suddenly, resulting in a heart rate outside the preset heart rate range, but this condition is normal and cannot be considered as unhealthy. Therefore, the first preset time period here may be 1 day, 3 days, one week, or the like. In the embodiment of the application, if the heart rate of each period is outside the preset heart rate range or the number of the periods with the heart rate outside the preset heart rate range exceeds the preset number of the periods in the first preset time period, the current heart rate health state of the user is judged to be in the unhealthy state, and the heart rate health state of the user can be timely and accurately determined.

Optionally, under the condition that the current heart rate health condition of the user is in a non-health state, outputting first prompt information, wherein the first prompt information is used for prompting the user to seek medical attention in time.

Optionally, under the condition that the current heart rate health condition of the user is in the unhealthy state, outputting the first prompt information may include: and under the condition that the current heart rate health condition of the user is judged to be in an unhealthy state, outputting first prompt information through a loudspeaker and/or outputting the first prompt information through a display interface.

Optionally, under the condition that the current heart rate health condition of the user is in the unhealthy state, outputting the first prompt information through the speaker may include: and outputting first prompt information through a loudspeaker under the conditions that the current heart rate health condition of the user is judged to be in an unhealthy state and the terminal equipment is detected to be inserted into the earphone. Therefore, other surrounding users cannot be interfered by the externally-played sound, and the privacy of the users can be protected.

Optionally, the vital sign data includes respiration, and the terminal device determines the current health condition of the user according to the vital sign data, and may include: judging that the current respiratory health condition of the user is in a health state under the condition that the respiration is in a preset respiratory range; and under the condition that the breath is out of the preset breath range, judging that the current breath health condition of the user is in a non-health state.

Optionally, the method may further include: and in the second preset duration, judging that the current respiratory health state of the user is in an unhealthy state under the condition that the respiratory of each period is outside a preset respiratory range or the number of the periods with the respiratory outside the preset respiratory range exceeds the number of the preset periods. It will be appreciated that the second preset duration of the present application is generally understood to be data having a sufficient duration, otherwise not of high reliability, for example if the preset duration is 1s, it is possible that the user sits for a long time, stands up suddenly, resulting in breathing outside the preset breathing range, but this condition is normal and cannot be considered as unhealthy. Therefore, the second preset time period here may be 1 day, 3 days, one week, or the like. In the embodiment of the application, if the respiration of each period is outside the preset respiration range or the number of the periods with the respiration outside the preset respiration range exceeds the preset number of the periods within the second preset duration, the current respiration health state of the user is judged to be in a non-health state, and the respiration health state of the user can be timely and accurately determined.

Optionally, under the condition that the current respiratory health condition of the user is in a non-health state, outputting second prompt information, wherein the second prompt information is used for prompting the user to seek medical attention in time.

Optionally, outputting the second prompting information under the condition that the current respiratory health condition of the user is judged to be in the unhealthy state may include: outputting the second prompt information through the loudspeaker and/or outputting the second prompt information through the display interface under the condition that the current respiratory health condition of the user is judged to be in the unhealthy state.

Optionally, outputting, by the speaker, the second prompt message when it is determined that the current respiratory health condition of the user is in the unhealthy state may include: and outputting second prompt information through a loudspeaker under the conditions that the current respiratory health condition of the user is judged to be in a non-health state and the terminal equipment is detected to be inserted into the earphone. Therefore, other surrounding users cannot be interfered by the externally-played sound, and the privacy of the users can be protected.

405. And closing the radar system switch under the condition that the vital sign data is unchanged within a preset duration.

It will be appreciated that the radar system switch may be turned off in the event that the vital sign data has not changed for a predetermined period of time. The method indicates that the distance between the user and the terminal equipment is far, and the vital sign data cannot be detected, so that unnecessary power consumption can be reduced when the radar system switch is closed.

Optionally, closing the radar system switch when the vital sign data is unchanged within a preset duration may include: and under the condition that the vital sign data is unchanged within a preset time period and the residual electric quantity of the terminal equipment is smaller than an electric quantity threshold value, the radar system switch can be turned off. The method has the advantages that the distance between the user and the terminal equipment is long, vital sign data cannot be detected, and unnecessary power consumption can be reduced and the operation of necessary application programs can be ensured when the radar system switch is turned off under the condition that the residual electric quantity of the terminal equipment is limited.

Optionally, the method may further include: and under the condition that the residual electric quantity of the terminal equipment is smaller than the electric quantity threshold value, the radar system switch can be closed. In order to close the switch of the radar system under the condition that the residual electric quantity of the terminal equipment is limited, the embodiment of the application can reduce unnecessary power consumption and ensure the operation of necessary application programs.

Optionally, the method for closing the switch of the radar system may include: the radar system switch is automatically and manually closed. In case of failure of the automatic switch, the radar system switch can also be manually turned off; alternatively, the radar system switch may be selected to be turned off automatically if it is inconvenient for the user to manually switch the radar system switch.

Optionally, closing the radar system switch when the vital sign data is unchanged within a preset duration may include: and under the condition that the vital sign data is unchanged within a preset duration, responding to the operation of closing the virtual icon switch by a user, and closing the radar system.

Optionally, the operation of closing the virtual icon switch includes an operation of sliding (e.g., sliding from right to left, sliding from bottom to top, etc.) the virtual icon switch, an operation of double clicking (or clicking) the virtual icon switch, or an operation of inputting a password, etc.

Exemplary, as shown in fig. 5A, a flowchart of a method for acquiring vital sign data in an embodiment of the present application is shown. The terminal equipment responds to the operation of opening a radar system switch by a user, opens the radar system, starts working, continuously collects first original data in the surrounding environment, inputs the collected first original data into a first preset network model for processing, and displays the obtained vital sign data on a display interface. The monitoring process can also be performed for a preset time period (for example, 10 s), if the detection finds that the vital sign data does not change within the preset time period (for example, 10 s), such as the user has left (for example, 1m which is not detected by the mobile phone) ³ Within) the radar system switch may also be actively turned off if the radar system switch is turned on, thereby reducing unnecessary power consumption. Thus, when the mobile phone is in the space near the user, after the millimeter wave switch is turned on, the health data (heartbeat/respiratory frequency) of the user can be displayed on the mobile phone in real time. Fig. 5B is a schematic diagram showing vital sign data according to an embodiment of the present application. In the embodiment of the application, whether the radar system module is closed or not is determined through continuous detection of the preset duration, so that the radar system switch can be closed and stopped without obvious physical change in the preset duration under the condition that the radar system switch is opened, the power consumption of a mobile phone can be reduced, and the cruising ability of terminal equipment is improved.

Optionally, the method may further include: and operating the terminal equipment according to at least one of face recognition, fingerprint recognition and voice recognition and combining the vital sign data.

Optionally, the operating the terminal device according to at least one of face recognition, fingerprint recognition, and voice recognition in combination with the vital sign data may include: and unlocking the terminal equipment according to at least one of face recognition, fingerprint recognition and voice recognition by combining the vital sign data.

An exemplary flowchart of unlocking by combining face recognition with vital sign data in the embodiment of the present application is shown in fig. 5C. I.e. vital sign data (e.g. heartbeat/respiration) detection functions can also enable other applications such as face recognition. Because the existing 2D face recognition always has a relatively difficult problem of anti-counterfeiting, for example, a photo is used to easily attack successfully, but the existing face recognition algorithm cannot be completely controlled, if the identification function of a millimeter wave true and false user is matched, the identification capability in the aspect can be improved, and in the face recognition unlocking process, if the current object is found to have no vital sign, the anti-counterfeiting can be directly reported and the unlocking is not performed. By combining the vital body characteristic data detection function, the existing 2D face recognition is enabled, and anti-counterfeiting problems such as photo/fake head attack and the like can be optimized.

Similarly, in the fingerprint identification and voice identification scenes, illegal persons can acquire fingerprints, voice recordings and the like of others by using some technologies, so that unlocking is performed, and property, data and the like are lost. If the current object is found to have no vital sign in the fingerprint identification and voice identification unlocking processes, the anti-counterfeiting can be directly reported and not unlocked.

Optionally, the method may further include: at least one of gesture detection and distance measurement is performed by the radar system. It will be appreciated that many aspects of the application may be extended by millimeter wave radar systems on terminal devices. I.e. in addition to being able to detect the heartbeat/respiration of the user, it can also be used for gesture detection (recognition), ranging etc., as long as there is some physical movement in the surrounding environment.

Optionally, the method may further include: responding to the selection operation of a user, and selecting a current working mode; in the case that the current operation mode is a vital sign data detection mode, steps 401 to 402 are performed, or steps 401 to 404 are performed, or steps 401 to 405 are performed;

and under the condition that the current working mode is a gesture detection mode and the radar system is in an on state, acquiring second original data through the radar system, inputting the second original data into a second preset network model, and outputting gesture data, wherein the second preset network model is an algorithm model obtained by performing model training according to a second historical original data set and a historical gesture data set. It should be noted that, the process of training the model according to the second historical original data set and the historical gesture data set is similar to the training process of the first preset network model, and will not be repeated here.

Fig. 5D is a schematic flow chart of heartbeat/respiration and gesture detection in the embodiment of the present application. After a radar system switch is turned on, a section of judging flow is added in the beginning stage, a working mode needing to be entered, a gesture detection mode or a vital sign data detection mode (namely a heartbeat/respiration detection mode) is selected firstly, then a subsequent processing flow is entered, original data are collected through the radar system, processing is carried out in the algorithm (namely a preset network model) processing stage by using the corresponding preset network model, a corresponding result is obtained and output, and then the result can be used for specific operation. For example, if the gesture is identified, the palm can be quickly swept over the mobile phone once to wake up the screen or operate the mobile phone screen at intervals, and a specific gesture can be set for unlocking at intervals and the like. Fig. 5E is a schematic diagram of acquiring second raw data of a gesture detection mode by the radar system according to an embodiment of the present application.

Optionally, whether gesture data or vital sign data detected within a preset time period change or not can be judged, and under the condition that the gesture data or vital sign data does not change, a radar system switch is turned off.

In the embodiment of the application, the method for acquiring vital sign data is applied to a terminal device, wherein the terminal device comprises a radar system, and the method comprises the following steps: under the condition that the radar system is in an on state, acquiring first original data through the radar system; inputting the first original data into a first preset network model, and outputting vital sign data, wherein the first preset network model is an algorithm model obtained by model training according to a first historical original data set and a historical vital sign data set. Because the terminal equipment comprises the radar system, the original data can be directly acquired through the radar system, and the original data is processed by the preset network model without direct contact with a user, so that the vital sign data is obtained, and the convenience for acquiring the vital sign data of the user is greatly improved. The low-power consumption millimeter wave radar system is integrated on the terminal equipment, so that physical contact is not needed, and at least one of heartbeat and respiration of a user can be conveniently detected in real time only by approaching the radar system, and the psychological change of emotion of the user can be recorded; at the same time, health problems in the aspect of the heart can be prevented, so that the functional diversity of the terminal equipment is increased.

As shown in fig. 6A, an embodiment of an apparatus for acquiring vital sign data in an embodiment of the present application is shown, where the apparatus is applied to a terminal device, and the terminal device includes a radar system, and the apparatus includes:

the acquisition module 601 is configured to acquire first raw data by the radar system when the radar system is in an on state;

the processing module 602 is configured to input the first raw data into a first preset network model, and output vital sign data, where the first preset network model is an algorithm model obtained by performing model training according to a first historical raw data set and a historical vital sign data set.

Optionally, the apparatus further includes:

a display module 603 for displaying the vital sign data on a display interface;

the processing module 602 is further configured to determine a current health condition of the user according to the vital sign data.

Optionally, the processing module 602 is further configured to operate the terminal device in combination with the vital sign data according to at least one of face recognition, fingerprint recognition, and voice recognition.

Optionally, the processing module 602 is further configured to close the radar system switch when the vital sign data is unchanged within a preset period of time.

Optionally, the processing module 602 is further configured to select a current operation mode in response to a selection operation of a user; performing the steps of claim 1 in case the current operation mode is a vital sign data detection mode;

the acquisition module 601 is further configured to acquire second original data through the radar system when the current working mode is a gesture detection mode and the radar system is in an on state;

the processing module 602 is further configured to input the second raw data into a second preset network model, and output gesture data, where the second preset network model is an algorithm model obtained by performing model training according to a second historical raw data set and a historical gesture data set.

Optionally, the wavelength of the radar system is millimeter wave; the vital sign data includes: at least one of heart rate and respiration.

Optionally, the processing module 602 is further configured to obtain the historical raw data and the historical vital sign data; and performing model training on the historical original data and the historical vital sign data to obtain the preset network model.

As shown in fig. 6B, which is a schematic diagram of an embodiment of a terminal device in an embodiment of the present application, an apparatus for acquiring vital sign data as shown in fig. 6A may be included.

In the embodiment of the present application, the charging device is a terminal device, and the terminal device may be a Mobile Phone (Mobile Phone), a tablet computer (Pad), a computer with a wireless transceiving function, a Virtual Reality (VR) terminal device, an augmented Reality (Augmented Reality, AR) terminal device, a wireless terminal device in industrial control (industrial control), a wireless terminal device in unmanned driving (self driving), a wireless terminal device in remote medical (remote medium), a wireless terminal device in smart grid (smart grid), a wireless terminal device in transportation safety (transportation safety), a wireless terminal device in smart city (smart city), or a wireless terminal device in smart home (smart home), or the like.

By way of example, and not limitation, in embodiments of the present application, the terminal device may also be a wearable device. The wearable device can also be called as a wearable intelligent device, and is a generic name for intelligently designing daily wear by applying wearable technology and developing wearable devices, such as glasses, gloves, watches, clothes, shoes and the like. The wearable device is a portable device that is worn directly on the body or integrated into the clothing or accessories of the user. The wearable device is not only a hardware device, but also can realize a powerful function through software support, data interaction and cloud interaction. The generalized wearable intelligent device includes full functionality, large size, and may not rely on the smart phone to implement complete or partial functionality, such as: smart watches or smart glasses, etc., and focus on only certain types of application functions, and need to be used in combination with other devices, such as smart phones, for example, various smart bracelets, smart jewelry, etc. for physical sign monitoring.

Fig. 7 is a schematic diagram of another embodiment of a terminal device according to an embodiment of the present application. The following describes the components of the mobile phone in the terminal device specifically with reference to fig. 7:

the RF circuit 710 may be configured to receive and transmit signals during a message or a call, and specifically, receive downlink information of a base station and process the downlink information with the processor 780; in addition, the data of the design uplink is sent to the base station. Typically, the RF circuitry 710 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, and the like. In addition, the RF circuitry 710 may also communicate with networks and other devices via wireless communications. The wireless communications may use any communication standard or protocol including, but not limited to, global system for mobile communications (Global System of Mobile communication, GSM), general packet radio service (General Packet Radio Service, GPRS), code division multiple access (Code Division Multiple Access, CDMA), wideband code division multiple access (Wideband Code Division Multiple Access, WCDMA), long term evolution (Long Term Evolution, LTE), email, short message service (Short Messaging Service, SMS), and the like.

The memory 720 may be used to store software programs and modules, and the processor 780 performs various functional applications and data processing of the handset by running the software programs and modules stored in the memory 720. The memory 720 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, memory 720 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.

The input unit 730 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the handset. In particular, the input unit 730 may include a touch panel 731 and other input devices 732. The touch panel 731, also referred to as a touch screen, may collect touch operations thereon or thereabout by a user (e.g., operations of the user on or thereabout the touch panel 731 using any suitable object or accessory such as a finger, a stylus, etc.), and drive the corresponding connection device according to a predetermined program. Alternatively, the touch panel 731 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 780, and can receive commands from the processor 780 and execute them. In addition, the touch panel 731 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. The input unit 730 may include other input devices 732 in addition to the touch panel 731. In particular, the other input devices 732 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, mouse, joystick, etc.

The display unit 740 may be used to display information input by a user or information provided to the user and various menus of the mobile phone. The display unit 740 may include a display panel 741, and alternatively, the display panel 741 may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch panel 731 may cover the display panel 741, and when the touch panel 731 detects a touch operation thereon or thereabout, the touch operation is transferred to the processor 780 to determine the type of touch event, and then the processor 780 provides a corresponding visual output on the display panel 741 according to the type of touch event. Although in fig. 7, the touch panel 731 and the display panel 741 are two separate components to implement the input and output functions of the mobile phone, in some embodiments, the touch panel 731 and the display panel 741 may be integrated to implement the input and output functions of the mobile phone.

The handset may also include at least one sensor 750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 741 according to the brightness of ambient light, and the proximity sensor may turn off the display panel 741 and/or the backlight when the mobile phone moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the acceleration in all directions (generally three axes), and can detect the gravity and direction when stationary, and can be used for applications of recognizing the gesture of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and knocking), and the like; other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that may also be configured with the handset are not described in detail herein.

Audio circuitry 760, speaker 761, and microphone 762 may provide an audio interface between a user and a cell phone. The audio circuit 760 may transmit the received electrical signal converted from audio data to the speaker 761, and the electrical signal is converted into a sound signal by the speaker 761 to be output; on the other hand, microphone 762 converts the collected sound signals into electrical signals, which are received by audio circuit 760 and converted into audio data, which are processed by audio data output processor 780 for transmission to, for example, another cell phone via RF circuit 710 or for output to memory 720 for further processing.

Wi-Fi belongs to a short-distance wireless transmission technology, and a mobile phone can help a user to send and receive e-mails, browse webpages, access streaming media and the like through the Wi-Fi module 770, so that wireless broadband Internet access is provided for the user. Although fig. 7 shows Wi-Fi module 770, it is to be understood that it does not belong to the necessary constitution of the cell phone, and can be omitted entirely as required within the scope of not changing the essence of the invention.

The processor 780 is a control center of the mobile phone, connects various parts of the entire mobile phone using various interfaces and lines, and performs various functions and processes of the mobile phone by running or executing software programs and/or modules stored in the memory 720 and calling data stored in the memory 720, thereby performing overall monitoring of the mobile phone. Optionally, the processor 780 may include one or more processing units; preferably, the processor 780 may integrate an application processor that primarily processes operating systems, user interfaces, applications, etc., with a modem processor that primarily processes wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 780.

The handset further includes a power supply 790 (e.g., a battery) for powering the various components, which may preferably be logically connected to the processor 780 through a power management system, such as to provide for managing charging, discharging, and power consumption by the power management system.

Although not shown, the mobile phone may further include a camera, a bluetooth module, etc., which will not be described herein.

In the embodiment of the present application, the terminal device further includes:

a radar system 7100 for acquiring first raw data by the radar system in a case that the radar system is in an on state;

the processor 780 is configured to input the first raw data into a first preset network model, and output vital sign data, where the first preset network model is an algorithm model obtained by performing model training according to a first historical raw data set and a historical vital sign data set.

Optionally, a display unit 740 is configured to display the vital sign data on a display interface;

the processor 780 is further configured to determine a current health condition of the user according to the vital sign data.

Optionally, the processor 780 is further configured to operate the terminal device in combination with the vital sign data according to at least one of face recognition, fingerprint recognition, and voice recognition.

Optionally, the processor 780 is further configured to close the radar system switch when the vital sign data is unchanged within a preset duration.

Optionally, the processor 780 is further configured to select a current working mode in response to a selection operation by a user; performing the steps of claim 1 in case the current operation mode is a vital sign data detection mode;

the radar system 7100 is further configured to collect second original data through the radar system when the current working mode is a gesture detection mode and the radar system is in an on state;

the processor 780 is further configured to input the second original data into a second preset network model, and output gesture data, where the second preset network model is an algorithm model obtained by performing model training according to a second historical original data set and a historical gesture data set.

Optionally, the vital sign data includes: at least one of heart rate and respiration.

Optionally, the processor 780 is further configured to obtain the historical raw data and the historical vital sign data; and performing model training on the historical original data and the historical vital sign data to obtain the preset network model.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy Disk, a hard Disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The above embodiments are merely for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.

Claims

1. A method of acquiring vital sign data, the method being applied to a terminal device, the terminal device comprising a radar system, the method comprising:

under the condition that the radar system is in an on state, acquiring first original data through the radar system;

inputting the first original data into a first preset network model, and outputting vital sign data, wherein the first preset network model is an algorithm model obtained by model training according to a first historical original data set and a historical vital sign data set.

2. The method according to claim 1, wherein the method further comprises:

Displaying the vital sign data on a display interface;

and judging the current health condition of the user according to the vital sign data.

3. The method according to claim 1 or 2, characterized in that the method further comprises:

and operating the terminal equipment according to at least one of face recognition, fingerprint recognition and voice recognition and combining the vital sign data.

4. The method according to claim 1 or 2, characterized in that the method further comprises:

and closing the radar system switch under the condition that the vital sign data is unchanged within a preset duration.

5. The method according to claim 1 or 2, characterized in that the method further comprises:

responding to the selection operation of a user, and selecting a current working mode;

performing the steps of claim 1 in case the current operation mode is a vital sign data detection mode;

and under the condition that the current working mode is a gesture detection mode and the radar system is in an on state, acquiring second original data through the radar system, inputting the second original data into a second preset network model, and outputting gesture data, wherein the second preset network model is an algorithm model obtained by performing model training according to a second historical original data set and a historical gesture data set.

6. The method according to claim 1 or 2, characterized in that the wavelength of the radar system is millimeter wave;

the vital sign data includes: at least one of heart rate and respiration.

7. The method according to claim 1 or 2, characterized in that the method further comprises:

acquiring the first historical original data set and the historical vital sign data set;

and performing model training on the historical original data set and the historical vital sign data set to obtain the first preset network model.

8. An apparatus for acquiring vital sign data, the apparatus being applied to a terminal device, the terminal device comprising a radar system, the apparatus comprising:

9. A terminal device, comprising:

A memory storing executable program code;

a process coupled to the memory;

the processor being adapted to correspondingly perform the method of any one of claims 1-7.

10. A computer readable storage medium comprising instructions which, when run on a processor, cause the processor to perform the method of any of claims 1-7.