CN116473547A - Sleep gesture recognition method and device, electronic equipment and storage medium - Google Patents
Sleep gesture recognition method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116473547A CN116473547A CN202310487224.9A CN202310487224A CN116473547A CN 116473547 A CN116473547 A CN 116473547A CN 202310487224 A CN202310487224 A CN 202310487224A CN 116473547 A CN116473547 A CN 116473547A
- Authority
- CN
- China
- Prior art keywords
- echo signal
- current
- characteristic
- determining
- posture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000005855 radiation Effects 0.000 claims abstract description 12
- 230000008859 change Effects 0.000 claims abstract description 11
- 239000013598 vector Substances 0.000 claims description 57
- 238000010586 diagram Methods 0.000 claims description 28
- 239000011159 matrix material Substances 0.000 claims description 28
- 238000004590 computer program Methods 0.000 claims description 16
- 230000036544 posture Effects 0.000 description 116
- 238000004891 communication Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000000605 extraction Methods 0.000 description 5
- 241000282414 Homo sapiens Species 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 238000001914 filtration Methods 0.000 description 4
- 230000035807 sensation Effects 0.000 description 4
- 230000033001 locomotion Effects 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 238000001208 nuclear magnetic resonance pulse sequence Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000003860 sleep quality Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 208000004210 Pressure Ulcer Diseases 0.000 description 1
- 208000034972 Sudden Infant Death Diseases 0.000 description 1
- 206010042440 Sudden infant death syndrome Diseases 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002146 bilateral effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000000284 resting effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 201000002859 sleep apnea Diseases 0.000 description 1
- 208000019116 sleep disease Diseases 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/48—Other medical applications
- A61B5/4806—Sleep evaluation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Physics & Mathematics (AREA)
- Veterinary Medicine (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Physiology (AREA)
- Dentistry (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention discloses a sleep gesture recognition method, a sleep gesture recognition device, electronic equipment and a storage medium. Acquiring a first electromagnetic echo signal received by radar equipment within a first preset time period; wherein the radiation range of the radar apparatus includes a sleep area of the target object; determining a prone reference characteristic value and a supine reference characteristic value corresponding to the target object based on the first electromagnetic echo signal; when the gesture change of the target object after the first preset time period is detected, acquiring a second electromagnetic echo signal received by radar equipment in the second preset time period, and determining current gesture characteristic data corresponding to the target object based on the second electromagnetic echo signal; based on the current posture characteristic data, the prone reference characteristic value and the supine reference characteristic value, the current sleeping posture category corresponding to the target object is determined, so that the non-contact recognition of the sleeping posture is realized, the convenience of the sleeping posture recognition is improved, and the accuracy of the sleeping posture recognition is improved.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a sleep gesture recognition method, a sleep gesture recognition device, an electronic device, and a storage medium.
Background
Sleep is a process of resting and recovering physical strength, and is vital to human beings. And the quality of sleep and sleep disorders are related to specific body gestures during sleep. Recognition of sleep gestures may help monitor a person's sleep quality to further assess physical health. In addition, recognition of sleep posture plays an important role in preventing sudden infant death syndrome, caring for patients with pressure ulcers, assisting patients with sleep apnea, and the like, and thus, recognition of sleep posture is of great practical significance.
At present, various sensors are generally used for collecting posture state data of a user during sleeping, a neural network model is adopted for recognizing sleeping posture based on the collected posture state data, and training data of the neural network model come from different tested users.
However, this recognition method requires the user to wear various sensors, which may cause the user to feel a large foreign body sensation, thereby affecting sleep quality. In addition, the individual variability of different users is large, and the traditional neural network model method is used for recognition, so that the accuracy of sleep gesture recognition is low.
Disclosure of Invention
The embodiment of the invention provides a sleep gesture recognition method, a sleep gesture recognition device, electronic equipment and a storage medium, so that the sleep gesture can be recognized in a non-contact manner, the convenience of sleep gesture recognition is improved, and the accuracy of sleep gesture recognition is improved.
In a first aspect, the present invention provides a sleep gesture recognition method, the method comprising:
acquiring a first electromagnetic echo signal received by radar equipment within a first preset time period; wherein the radiation range of the radar apparatus includes a sleep area of the target object;
determining a prone reference characteristic value and a supine reference characteristic value corresponding to the target object based on the first electromagnetic echo signal;
when the gesture change of the target object after the first preset time period is detected, acquiring a second electromagnetic echo signal received by the radar equipment within a second preset time period, and determining current gesture characteristic data corresponding to the target object based on the second electromagnetic echo signal;
and determining the current sleeping posture category corresponding to the target object based on the current posture characteristic data, the prone reference characteristic value and the supine reference characteristic value.
In a second aspect, the present invention provides a sleep posture recognition apparatus, comprising:
The reference signal acquisition module is used for acquiring a first electromagnetic echo signal received by the radar equipment within a first preset time period; wherein the radiation range of the radar apparatus includes a sleep area of the target object;
the reference characteristic determining module is used for determining a prone reference characteristic value and a supine reference characteristic value corresponding to the target object based on the first electromagnetic echo signal;
the characteristic data determining module is used for acquiring a second electromagnetic echo signal received by the radar equipment within a second preset duration when the gesture change of the target object after the first preset duration is detected, and determining current gesture characteristic data corresponding to the target object based on the second electromagnetic echo signal;
and the current sleeping posture determining module is used for determining the current sleeping posture category corresponding to the target object based on the current posture characteristic data, the prone reference characteristic value and the supine reference characteristic value.
In a third aspect, the present invention provides an apparatus comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the sleep gesture recognition method of any one of the embodiments of the present invention.
In a fourth aspect, the present invention provides a computer readable storage medium storing computer instructions for causing a processor to perform the sleep gesture recognition method of any one of the embodiments of the present invention.
According to the technical scheme provided by the embodiment of the invention, the first electromagnetic echo signal received by the radar equipment in the first preset time period is obtained, wherein the radiation range of the radar equipment comprises the sleeping area of the target object, and the prone reference characteristic value and the supine reference characteristic value corresponding to the target object are determined based on the first electromagnetic echo signal, so that the personalized reference characteristic value of the target user can be obtained. When the gesture change of the target object after the first preset time period is detected, a second electromagnetic echo signal received by the radar equipment in the second preset time period is obtained, the current gesture characteristic data corresponding to the target object is determined based on the second electromagnetic echo signal, and then the current sleeping gesture category corresponding to the target object is determined based on the current gesture characteristic data, the prone reference characteristic value and the supine reference characteristic value. According to the method and the device, the technical problem that the accuracy rate of sleep gesture recognition based on the wearable sensor equipment is low is solved, a user does not need to wear various sensor equipment, foreign body sensation is not brought to the user, the non-contact recognition of the sleep gesture can be realized, the convenience of sleep gesture recognition is improved, and the accuracy rate of sleep gesture recognition is improved by acquiring the individualized sleep gesture feature reference value of the target object.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a sleep gesture recognition method according to a first embodiment of the present invention;
FIG. 2 is a schematic diagram of a radar apparatus according to a first embodiment of the present invention and a target object;
FIG. 3 is a schematic diagram of a radar echo signal according to a first embodiment of the present invention;
FIG. 4 is a pictorial view of a target feature in accordance with an embodiment of the present invention;
fig. 5 is a flowchart of a sleep gesture recognition method according to a second embodiment of the present invention;
fig. 6 is a schematic structural diagram of a sleep gesture recognition apparatus according to a third embodiment of the present invention;
Fig. 7 is a schematic structural diagram of an apparatus according to a fourth embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that, in the description and claims of the present invention and the above figures, the terms "first preset condition", "second preset condition", and the like are used to distinguish similar objects, and are not necessarily used to describe a specific order or precedence. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1 is a flowchart of a sleep gesture recognition method according to an embodiment of the present invention, where the embodiment is applicable to a situation of recognizing a sleep gesture of a target object according to an electromagnetic echo signal of a radar. The method may be performed by a sleep gesture recognition apparatus, which may be implemented in hardware and/or software, which may be configured on a computer device, which may be a notebook, desktop computer, smart tablet, etc. As shown in fig. 1, the method includes:
s110, acquiring a first electromagnetic echo signal received by the radar equipment within a first preset time period.
Wherein the radiation range of the radar apparatus includes a sleep area of the target object. The target object is a user who needs to perform sleep gesture recognition. For example, referring to fig. 2, a schematic diagram of a relative position of a radar apparatus and a target object is shown in fig. 2, the radar apparatus is installed at a position associated with a target bed, and the target object is laid on the target bed when performing sleep posture recognition on the target object. The electromagnetic echo signal is a reflected electromagnetic wave corresponding to the detected electromagnetic wave emitted by the radar device. The first preset duration is a preset time, for example, the first preset duration is 8 hours. The first electromagnetic echo signal is an electromagnetic echo signal of a first preset duration.
Specifically, the signal transmitter of the radar apparatus periodically transmits a detection electromagnetic wave signal to the sleep area of the target object, and the electromagnetic signal received by the signal receiver of the radar apparatus after the detection electromagnetic wave signal is scattered by the target object is an electromagnetic echo signal. In order to improve the accuracy of the result of the real-time sleeping posture detection of the target user, the sleeping posture characteristics corresponding to the target object can be learned in advance, and based on the sleeping posture characteristics, a first electromagnetic echo signal of the target user in a first preset duration is acquired before the real-time sleeping posture detection of the target object is carried out, so that the personalized data characteristics of the target user are extracted based on the first electromagnetic echo signal.
For example, the first preset duration is 8 hours, the target object starts to sleep in the radar device radiation area at 10 o 'clock on the first day and ends to sleep at 6 o' clock on the second day, and the electromagnetic echo data collected by the radar device in the period from 10 o 'clock on the first day to 6 o' clock on the second day is the first electromagnetic echo data.
In this embodiment, if the detected electromagnetic wave signal transmitted by the radar apparatus is a pulse signal, the transmitter of the radar apparatus may transmit a periodic pulse sequence, and the period may be defined as a pulse repetition interval. The receiver of the radar apparatus may receive a radar echo signal corresponding to the detected electromagnetic wave signal. The radar echo signal is shown in fig. 3, and as shown in fig. 3, the radar echo signal and the detected electromagnetic wave signal are in one-to-one correspondence, so that the radar echo signal is also a periodic pulse sequence. The pulse sequence of each radar echo signal is stored separately in rows, e.g. a first pulse of the radar echo signal is placed in a first row, a second pulse of the same radar echo signal is placed in a second row, and so on. In general, the dimension as seen in the direction of the rows is defined as the fast time dimension, and in addition, the dimension as seen in the direction of the columns is defined as the slow time dimension since the data sampling interval between rows tends to be greater than the pulse duration.
In this embodiment, after the electromagnetic echo signal received by the radar device is obtained, the electromagnetic echo signal needs to be preprocessed to remove clutter in the electromagnetic echo signal, and a clean electromagnetic echo signal is reserved. Based on the method, firstly, fourier transformation is carried out on the received electromagnetic echo signals in a fast time dimension, and then static clutter suppression is carried out to obtain pure electromagnetic echo signals. In the subsequent processing, the processing is performed based on the electromagnetic echo signals after the preprocessing, and thus, in the subsequent embodiment, the electromagnetic echo signals mentioned are all electromagnetic echo signals after the preprocessing.
And S120, determining a prone reference characteristic value and a supine reference characteristic value corresponding to the target object based on the first electromagnetic echo signal.
Wherein the prone reference feature value is a feature value that the target object sleep posture is a prone posture. The supine reference feature value is a feature value in which the sleep posture of the target subject is the supine posture.
Specifically, in a first preset duration, the target user corresponds to a plurality of different sleeping postures, so that the first electromagnetic echo signal is firstly divided into a plurality of electromagnetic echo signal segments based on the moment when the user turns over, and posture characteristic data corresponding to each electromagnetic echo data segment are extracted. For each electromagnetic echo data segment, whether or not the sleep posture corresponding to this signal segment is a lateral posture is determined based on lateral characteristic data for distinguishing the lateral posture from the non-lateral posture in the posture characteristic data. Further, a signal segment of the first electromagnetic echo signal, the sleep posture of which is not the lateral lying posture, is extracted as a pitching echo signal segment, and then the pitching echo signal segment is subjected to feature classification, so that prone data features corresponding to the sleep posture signal segment and supine data features corresponding to the supine posture signal segment are determined. To determine a prone reference feature value based on prone data features and to determine a supine reference feature value based on supine data features.
In the present embodiment, the manner of determining the posture feature data is the same for each electromagnetic echo signal segment, and one of the electromagnetic echo signal segments is exemplified here. To extract gesture feature data corresponding to electromagnetic echo signal segments, a target feature map corresponding to the electromagnetic echo signal segments is first determined. The target feature map is used for representing the corresponding relation among the distance between the target object and the radar equipment, the Doppler frequency value and the signal amplitude value, so the target feature map is also called a distance Doppler map. For electromagnetic echo signal segments, the target feature map may be determined by fourier transforming along a slow time dimension to obtain the target feature map. The electromagnetic echo signals may be characterized by a matrix, which may be referred to as a radar echo matrix, the row vectors of which correspond to the fast time dimension and the column vectors to the slow time dimension. And performing Fourier transform on the radar echo matrix along the direction of the slow time dimension, namely performing Fourier transform on each column vector of the radar echo matrix respectively, so as to obtain a Fourier transform matrix. The fourier transform formula is as follows:
where x (k, n) is the data in one time window, one time window is one column vector, so x (k, n) is the data of one column vector. K e [1, K ], K is the length of the time window, ω is the transformation frequency, and w (m) is the Hamming window function.
The dimension of the fourier transform matrix obtained after fourier transform is the same as the dimension of the radar echo matrix. For example, the radar echo matrix is a 1000×500-dimensional matrix, and the fourier transform matrix is also a 1000×500-dimensional matrix. Further, the fourier transform matrix is converted into a target feature map by a mapping program. The corresponding numerical value of each feature point in the target feature map is the amplitude of each element of the Fourier transform matrix. After the target feature map is generated, denoising the target feature map to obtain a pure target feature map with higher signal-to-noise ratio, wherein the pure target feature map can be expressed asHere, the denoising process may employ mean filtering, gaussian filtering, median filtering, bilateral filtering, or the like, which is not particularly limited herein.
In the target feature map, the abscissa is the distance between the target object and the radar device, the ordinate is the doppler frequency value, the feature points in the target feature map are magnitudes corresponding to fourier transform matrix elements, and an exemplary generated target feature map is shown in fig. 4, where (a) is a target feature map corresponding to a supine posture, (B) is a target feature map corresponding to a lateral posture, and (C) is a target feature map corresponding to a lateral posture. In practical application, the target feature map is a color image, and the target feature map is composed of a large number of feature points, each feature point corresponds to different amplitude values, the larger the amplitude value is, the more red the color of the corresponding feature point is, and the smaller the amplitude value is, the more blue the color of the corresponding feature point is.
After the target feature map is obtained, the function expression form corresponding to the gesture feature of the target feature map is further extracted, and the amplitude information of the target feature map and the function expression column vector amplitude information G corresponding to the shape information can be extracted 1 (n) first shape characteristic information G 2 (n) second shape characteristic information G 3 (n), line vector magnitude information G 4 (eta). And further based on specific feature data and corresponding function expression form G of gesture features in the target feature map 1 (n)、G 2 (n)、G 3 (n)、G 4 (eta) determinationAnd determining a plurality of gesture characteristic data corresponding to the electromagnetic echo signal segments. The determined gesture feature data may include, but is not limited to: second non-zero column feature point amplitude F 1 Third non-zero column characteristic point amplitude F 2 Maximum amplitude F of second-largest non-zero column feature point 3 And the third maximum amplitude F of non-zero column feature points 4 Second non-zero column header size F 5 Third non-zero column header size F 6 First threshold jog frequency distribution F 7 Second threshold jog frequency distribution F 8 Amplitude distribution F of first column number 9 Second column number amplitude distribution F 10 And body trunk maximum Doppler frequency F 11 . Wherein F is 1 -F 6 Is the feature data of lateral lying; f (F) 7 -F 11 Is pitch and horizontal characteristic data.
S130, when the gesture change of the target object after the first preset time period is detected, acquiring a second electromagnetic echo signal received by the radar equipment within a second preset time period, and determining current gesture feature data corresponding to the target object based on the second electromagnetic echo signal.
The second preset time length is used for distinguishing and expressing the first preset time length. The second preset time period is a preset time period, for example, the second preset time period is 1 minute. The second electromagnetic echo signal is an electromagnetic echo signal of a second preset duration. The current gesture feature data is used for representing a specific feature quantity corresponding to the sleeping gesture of the target object at the current moment.
Specifically, after a first preset duration, the personalized reference feature of the target object is already acquired. And when the gesture change of the target object is detected, acquiring a second electromagnetic echo signal within a second preset time period from the moment of the gesture change. And generating a current target feature map according to the second electromagnetic echo signal, extracting current amplitude information and current shape information from the current target feature map, and further determining current gesture feature data corresponding to the target object based on the current amplitude information and the current shape information.
And S140, determining the current sleeping posture category corresponding to the target object based on the current posture characteristic data, the prone reference characteristic value and the supine reference characteristic value.
The current sleeping posture category comprises a lateral lying posture category, a prone lying posture category or a supine lying posture category. The current posture feature data includes current lateral feature data and current pitch-lie feature data.
Optionally, S140 specifically includes the following steps: based on the current lateral lying characteristic data, determining whether the current sleeping posture category corresponding to the target object is a lateral lying posture category; if not, determining the current sleeping posture category corresponding to the target object based on the current pitching feature data, the prone reference feature value and the supine reference feature value.
Specifically, based on the current lateral-lying characteristic data in the current posture characteristic data and a pre-trained lateral-lying classifier, whether the sleeping posture corresponding to the target object at the current moment is a lateral-lying posture or a non-lateral-lying posture is judged. If the result output by the side lying classifier shows that the sleeping posture corresponding to the target object at the current moment is the side lying posture, the current sleeping posture class corresponding to the target object is the side lying posture class. If the result output by the side lying classifier shows that the sleeping posture corresponding to the target object at the current moment is a non-side lying posture, the sleeping posture corresponding to the target object at the current moment needs to be determined to be a prone posture type or a supine posture type. The specific implementation process can be as follows: calculating Euclidean distance between the current prone feature data and the prone reference feature value, so as to determine first similarity between the current pitching feature data and the prone reference feature value; and calculating Euclidean distance between the current prone supine characteristic data and the supine reference characteristic value, so as to determine second similarity between the current pitching characteristic data and the supine reference characteristic value, and determining whether the current sleeping posture category corresponding to the target object is a prone posture category or a supine posture category based on the first similarity and the second similarity.
In another embodiment, based on the current lateral-lying characteristic data in the current posture characteristic data, whether the sleep posture corresponding to the target object at the current moment is the lateral-lying posture or the non-lateral-lying posture is determined by voting. For example, if the second electromagnetic echo signal is obtained for 1 minute, whether the sleep posture corresponding to the second electromagnetic echo signal is a lateral posture or a non-lateral posture may be determined based on the current lateral characteristic data corresponding to the second electromagnetic echo signal per second, and further whether the sleep posture corresponding to the target object is a lateral posture may be determined by voting based on the prediction result of 60 times. For example, if the result of 40 of the 60 predictions is a lateral posture, the sleep posture corresponding to the subject is a lateral posture type.
According to the technical scheme provided by the embodiment of the invention, the first electromagnetic echo signal received by the radar equipment in the first preset time period is obtained, wherein the radiation range of the radar equipment comprises the sleeping area of the target object, and the prone reference characteristic value and the supine reference characteristic value corresponding to the target object are determined based on the first electromagnetic echo signal, so that the personalized reference characteristic value of the target user can be obtained. When the gesture change of the target object after the first preset time period is detected, a second electromagnetic echo signal received by the radar equipment in the second preset time period is obtained, the current gesture characteristic data corresponding to the target object is determined based on the second electromagnetic echo signal, and then the current sleeping gesture category corresponding to the target object is determined based on the current gesture characteristic data, the prone reference characteristic value and the supine reference characteristic value. According to the method and the device, the technical problem that the accuracy rate of sleep gesture recognition based on the wearable sensor equipment is low is solved, a user does not need to wear various sensor equipment, foreign body sensation is not brought to the user, the non-contact recognition of the sleep gesture can be realized, the convenience of sleep gesture recognition is improved, and the accuracy rate of sleep gesture recognition is improved by acquiring the individualized sleep gesture feature reference value of the target object.
Example two
Fig. 5 is a flowchart of a sleep posture recognition method according to a second embodiment of the present invention, where on the basis of the foregoing embodiments, the steps of "determining a prone reference feature value and a supine reference feature value corresponding to the target object based on the first electromagnetic echo signal" and "determining current posture feature data corresponding to the target object based on the second electromagnetic echo signal" are further optimized, and the embodiments of the present invention may be combined with each of the alternatives in the foregoing one or more embodiments. As shown in fig. 5, the method includes:
s210, acquiring a first electromagnetic echo signal received by radar equipment within a first preset time period; wherein the radiation range of the radar apparatus includes a sleep area of the target object.
S220, dividing the first electromagnetic echo signal into at least one electromagnetic echo signal segment, and determining posture characteristic data of each electromagnetic echo signal segment.
Wherein the posture feature data includes lateral and pitch feature data. On the basis of the embodiment, the gesture feature data includes a second non-zero column feature point amplitude F 1 Third non-zero column characteristic point amplitude F 2 Maximum amplitude F of second-largest non-zero column feature point 3 And the third maximum amplitude F of non-zero column feature points 4 Second non-zero column header size F 5 Third non-zero column header size F 6 First threshold jog frequency distribution F 7 Second threshold jog frequency distribution F 8 Amplitude distribution F of first column number 9 Second column number amplitude distribution F 10 And body trunk maximum Doppler frequency F 11 。F 1 -F 6 Is the feature data of lateral lying; f (F) 7 -F 11 Is pitch and horizontal characteristic data.
In this embodiment, dividing the first electromagnetic echo signal into at least one electromagnetic echo signal segment may be understood as a process of separating human body states, that is, two human body states of "motion" and "lying" may be separated by body movement index estimation, and the "motion" is taken as a division point, and the first electromagnetic echo signal data is divided into a plurality of electromagnetic echo signal segments, where the human body maintains the same posture in one segment of data.
Optionally, determining pose characteristic data of each electromagnetic echo signal segment in step S220 specifically includes the following steps:
(1) For each electromagnetic echo signal segment, generating a target feature map based on the electromagnetic echo signals;
in this embodiment, the specific implementation process of step (1) is already described in detail in the above embodiments, and will not be described herein.
(2) Extracting amplitude information and shape information in a target feature map;
how the amplitude information and the shape information of the image are extracted from the target feature map is described in detail next. The amplitude information specifically includes column vector amplitude information G 1 (n) and row vector magnitude information G 4 (eta). The shape information specifically includes first shape characteristic information G 3 (n) and second shape characteristic information G 3 (n)。
Optionally, extracting the amplitude information in the target feature map specifically includes: determining the column characteristic amplitude corresponding to each column vector based on the amplitude value corresponding to each element in each column vector in the target characteristic diagram, and determining the column vector amplitude information of the target characteristic diagram based on each column characteristic amplitude; and determining the row characteristic amplitude corresponding to each column vector based on the amplitude value corresponding to each element in each row vector in the target characteristic diagram, and determining the row vector amplitude information of the target characteristic diagram based on each row characteristic amplitude.
In this embodiment, for each column vector in the target feature map, the amplitude values corresponding to the elements in the column vector are added as the column feature amplitudes corresponding to the column vector, and the array of the column feature amplitudes is used as the column vector amplitude information of the target feature map. Similarly, for the row vector amplitude information in the target feature map, the amplitude values corresponding to the elements in the row vector are added to be the row feature amplitude corresponding to the row vector, and the array formed by the row feature amplitude information is used as the row vector amplitude information of the target feature map. If the target feature map is expressed as The column vector magnitude information of the target feature map can be expressed as: />The row vector magnitude information of the target feature map can be expressed as:
for example, if the fourier transform matrix corresponding to the target feature map is expressed as:
| 0 | 0 | 0 | 0 | 0 | 0 |
| 0 | 1 | 2 | 0 | 0 | 0 |
| 0 | 2 | 3 | 3 | 3 | 3 |
| 0 | 3 | 4 | 4 | 2 | 4 |
| 0 | 0 | 5 | 6 | 5 | 5 |
| 0 | 2 | 9 | 8 | 7 | 5 |
| 0 | 1 | 6 | 6 | 6 | 8 |
| 0 | 3 | 0 | 0 | 0 | 0 |
| 0 | 0 | 0 | 0 | 0 | 0 |
| 0 | 0 | 0 | 0 | 0 | 0 |
on the basis of the above exemplary embodiment, the summation of the amplitude values corresponding to all the elements in each column vector is respectively: 0. 12, 29, 27, 23, 25, and 0, the column vector magnitude information of the target feature map may represent: g 1 (n) = {0, 12, 29, 27, 23, 25}. The summation of amplitude values corresponding to all elements in each row of vectors is as follows: 0. 3, 14, 17, 21, 31, 27, 3, 0, and 0, the row vector magnitude information of the target feature map may represent: g 4 (η)＝{0，3，14，17，21，31，27，3，0，0}。
Optionally, extracting shape information in the target feature map specifically includes the following steps:
1) And determining the shape characteristic value corresponding to each element based on the amplitude value corresponding to each element in the target characteristic diagram and a preset amplitude threshold value.
The preset amplitude threshold is a preset amplitude value, for example, the preset amplitude threshold is 0.
In this embodiment, the magnitude value corresponding to each element in the target feature map is compared with a preset magnitude threshold. If the amplitude value corresponding to the element in the target feature diagram is larger than a preset amplitude threshold, the shape feature value corresponding to the element position is a line number; if the amplitude value corresponding to the element in the target feature map is smaller than the preset amplitude threshold, the shape feature value corresponding to the element position is zero. For example, assuming that the preset amplitude threshold is 0, the shape feature value corresponding to each element in the target feature map may be expressed as:
2) And determining a shape characteristic matrix composed of shape characteristic values, performing norm operation on each column vector in the shape characteristic matrix, and determining first shape characteristic information corresponding to each column.
On the basis of the above exemplary method, after determining the shape feature value corresponding to each element in the target feature map, a new matrix can be obtained, and this matrix is used as the shape feature matrix. If P n ＝[Q(1,n),Q(2,n),…,Q(H,n),] T The corresponding first shape characteristic information of each column may be expressed as G 2 (n)＝||P n || 0 . The first shape characteristic information is used to characterize how many points in each column vector are valued.
3) And determining second shape characteristic information corresponding to each column based on the maximum shape characteristic value and the minimum shape characteristic value of each column in the shape characteristic matrix.
In the present embodiment, the maximum shape feature value max of each column vector in the shape feature matrix is determined η Q (η, n) and minimum shape feature value min η Q (η, n), the second shape characteristic information corresponding to each column may be expressed as: g 3 (n)＝max η Q(η,n)-min η Q(η,n)。
(3) Based on the amplitude information and the shape information, pose characteristic data of the electromagnetic echo signal segment is determined.
In the present embodiment, in the extracted amplitude information G 1 (n) and G 4 (eta) and shape information G 2 (n) and G 3 After (n), it can be determined according to G 1 (n)、G 2 (n)、G 3 (n)、G 4 (eta) extracting 10 pieces of gesture feature data. The extracted 10 pieces of posture feature data may be divided into 4 groups, and how the 10 pieces of posture feature data are determined is described in detail below. In addition, the gesture feature data is other than according to G 1 (n)、G 2 (n)、G 3 (n)、G 4 (eta) the maximum Doppler frequency of the torso is included in addition to the 10 features extracted.
A first group: the strong characteristic point amplitude characteristic of different body parts comprises a second non-zero column characteristic point amplitude F 1 Third non-zero column characteristic point amplitude F 2 Maximum amplitude F of second-largest non-zero column feature point 3 And the third largest non-zero column feature pointLarge amplitude F 4 The specific extraction formula is as follows:
wherein N is 2 For the second non-zero column vector in the target feature map, N 3 And is the third non-zero column vector in the target feature map.G respectively 1 The second and third largest magnitudes of (n). Based on the above exemplary embodiment, if G 1 (n) = {0, 12, 29, 27, 23, 25}, then +.>27->25.
Second group: the header size feature includes a second non-zero column header size F 5 And a third non-zero column header size F 6 The specific extraction formula is as follows:
wherein the vector isVector->Here Γ (l) and Θ (l) are respectively represented as follows:
wherein epsilon is a threshold value, G 2m 、G 3m G respectively 2 (l)、G 3 (l) Is G, l' are each 2m 、G 3m The corresponding column number.
Third group: the body jog frequency profile features include a first threshold jog frequency profile F 7 And a second threshold jog frequency profile F 8 The specific extraction formula is as follows:
F 7 ＝||C 1 || 0 (11)
F 8 ＝||C 2 || 0 (12)
wherein C is 1 、C 2 Respectively represent vectors [ W (1), W (2), …, W (L)]The threshold zeta of (1) is respectively zeta 1 、ζ 2 (ζ 1 <ζ 2 ) Vector in time. W (η) here is represented as follows:
fourth group: the body characteristic point amplitude distribution characteristic comprises a first column number amplitude distribution F 9 And a second column number amplitude distribution F 10 The specific extraction formula is as follows:
wherein N is 0 The column number corresponding to the first non-zero column vector in the target feature diagram is the nearest column number of the target object to the radar, n 1 、n 2 Representing the number of column vectors.
In particular, when determining the posture feature data, the body trunk maximum doppler frequency may be determined, and the body trunk maximum doppler frequency does not need to be determined from the amplitude information and the shape information, but the body trunk maximum doppler frequency will be described here for the purpose of explaining all the data included in the posture feature data. The extraction formula corresponding to the maximum Doppler frequency of the trunk of the body is as follows:
F 11 ＝max(|f +max |,|f -max |) (16)
wherein f +max Maximum value of positive Doppler frequency in target characteristic diagram, f -max The maximum value of the negative doppler frequency in the target profile.
S230, for each electromagnetic echo signal segment, determining whether the sleeping posture corresponding to the electromagnetic echo signal segment is a lateral posture or not based on the lateral characteristic data.
In this embodiment, whether the sleep posture corresponding to the electromagnetic echo signal segment is the lateral posture or not is determined by voting. That is, each electromagnetic echo signal segment can be divided into a plurality of sub-data segments, and each sub-data segment is respectively based on the lateral characteristic data and a pre-trained lateral classifier to judge whether the sleeping posture category corresponding to the sub-data segment is a lateral posture category or a non-lateral posture category. Further, by comparing the number of the side-lying posture type sub-data segments with the number of the non-side-lying posture type sub-data segments, it is determined whether or not the sleep posture corresponding to the electromagnetic echo signal segment is the side-lying posture.
S240, removing the electromagnetic echo signal segment with the sleeping posture being the lateral posture from at least one electromagnetic echo signal segment to obtain the pitching echo signal segment.
On the basis of S230, the electromagnetic echo signal segments of the first electromagnetic echo signal, which have been determined to be of the lateral posture type, are removed, only the electromagnetic echo signal segments of the non-lateral posture type are retained, and these electromagnetic echo signal segments are regarded as pitch-lying echo signal segments.
S250, feature sorting is carried out on the pitching and lying feature data corresponding to each pitching and lying echo signal segment, and prone reference feature values and supine reference feature values corresponding to the target object are determined.
In the present embodiment, the manner of feature sorting of the pitch-lying feature data is not particularly limited, as long as it is determined that the sleep posture can be given as the prone-lying data feature corresponding to the prone posture and the sleep posture can be given as the supine-lying data feature corresponding to the supine posture. Further, a prone reference feature value is determined based on the prone data feature, and a supine reference feature value is determined based on the supine data feature.
And S260, generating a current target feature map based on the second electromagnetic echo signal, and extracting current amplitude information and current shape information in the current target feature map.
In this embodiment, the manner of generating the current target feature map based on the second electromagnetic echo signal is the same as that of generating the target feature map based on the first electromagnetic echo signal in the above embodiment, and will not be described herein. After the current target feature map is generated, the manner in which the current amplitude information and the current shape information are extracted therefrom is the same as "extracting the amplitude information and the shape information in the target feature map" in step S220. Can extract the current column vector amplitude information H 1 (n) current first shape characteristic information H 2 (n) current second shape characteristic information H 3 (n) current line vector magnitude information H 4 (η)。
S270, determining current gesture feature data corresponding to the target object based on the current amplitude information and the current shape information.
In the present embodiment, the specific feature data in the current target feature map and H are based on 1 (n)、H 2 (n)、H 3 (n)、H 4 (eta) the current gesture feature data corresponding to the target object may be determined. The determined current gesture feature data may include, but is not limited to: current second non-zero column feature point amplitude M 1 Current third non-zero column feature point amplitude M 2 Maximum amplitude M of current second largest non-zero column feature point 3 And the current third maximum amplitude M of non-zero column feature points 4 Current second non-zero column header size M 5 Current third non-zero column header size M 6 Current first threshold jog frequency profile M 7 Current second threshold jog frequency profile M 8 Current first column number amplitude distribution M 9 Current second column number amplitude distribution M 10 And the current body torso maximum Doppler frequency M 11 。
S280, determining the current sleeping posture category corresponding to the target object based on the current posture characteristic data, the prone reference characteristic value and the supine reference characteristic value.
According to the technical scheme provided by the embodiment of the invention, when the prone reference characteristic value and the supine reference characteristic corresponding to the target object are determined, the target characteristic diagram is firstly generated based on the first electromagnetic echo signal, and then the amplitude information and the shape information in the target characteristic diagram are extracted, so that the gesture characteristic data of the electromagnetic echo signal section are determined based on the amplitude information and the shape information, and the prone reference characteristic value and the supine reference characteristic value corresponding to the target object are determined based on the gesture characteristic data. According to the embodiment of the invention, after the first electromagnetic echo signal is acquired, the target feature map corresponding to the first electromagnetic echo signal is determined, the electromagnetic echo signal can be converted into an image form, the sleeping gesture features extracted from the image are more than the information quantity of the sleeping gesture features extracted from the pulse signal directly, in general, the more the feature quantity of the sleeping gesture can be distinguished, the higher the accuracy of recognizing the sleeping gesture is, and therefore the accuracy of recognizing the sleeping gesture can be improved.
Example III
Fig. 6 is a schematic structural diagram of a sleep gesture recognition apparatus according to a third embodiment of the present invention, where the apparatus may perform the sleep gesture recognition method according to the embodiment of the present invention. The device comprises: a reference signal acquisition module 310, a reference feature determination module 320, a feature data determination module 330, and a current sleep pose determination module 340.
A reference signal acquisition module 310, configured to acquire a first electromagnetic echo signal received by a radar device within a first preset duration; wherein the radiation range of the radar apparatus includes a sleep area of the target object;
a reference feature determining module 320, configured to determine a prone reference feature value and a supine reference feature value corresponding to the target object based on the first electromagnetic echo signal;
the feature data determining module 330 is configured to, when detecting that the pose of the target object changes after the first preset duration, obtain a second electromagnetic echo signal received by the radar device within a second preset duration, and determine current pose feature data corresponding to the target object based on the second electromagnetic echo signal;
the current sleeping posture determining module 340 is configured to determine a current sleeping posture category corresponding to the target object based on the current posture feature data, the prone reference feature value, and the supine reference feature value.
Based on the above aspects, the reference feature determining module 320 includes:
the sleeping posture reference characteristic determining submodule is used for dividing the first electromagnetic echo signal into at least one electromagnetic echo signal segment and determining posture characteristic data of each electromagnetic echo signal segment; wherein the posture feature data includes lateral and pitch feature data;
The side lying posture judging sub-module is used for determining whether the sleeping posture corresponding to each electromagnetic echo signal segment is a side lying posture or not based on the side lying characteristic data;
the prone position data segment obtaining submodule is used for removing an electromagnetic echo signal segment with a sleeping posture being a lateral position from the at least one electromagnetic echo signal segment to obtain a pitching echo signal segment;
and the reference characteristic value determining submodule is used for carrying out characteristic sorting on the pitching characteristic data corresponding to each pitching echo signal segment and determining a prone reference characteristic value and a supine reference characteristic value corresponding to the target object.
On the basis of the technical schemes, the sleeping posture reference characteristic determining submodule comprises:
a feature map determining unit configured to generate, for each of the electromagnetic echo signal segments, a target feature map based on electromagnetic echo signals;
a feature information acquisition unit for extracting amplitude information and shape information in the target feature map;
and the characteristic data acquisition unit is used for determining gesture characteristic data of the electromagnetic echo signal segment based on the amplitude information and the shape information.
On the basis of the above technical solutions, the feature information obtaining unit includes:
The amplitude information acquisition subunit is used for determining the column characteristic amplitude corresponding to each column vector based on the amplitude value corresponding to each element in each column vector in the target characteristic diagram, and determining the column vector amplitude information of the target characteristic diagram based on each column characteristic amplitude; and determining the row characteristic amplitude corresponding to each column vector based on the amplitude value corresponding to each element in each row vector in the target characteristic diagram, and determining the row vector amplitude information of the target characteristic diagram based on each row characteristic amplitude.
A shape information obtaining subunit, configured to determine a shape feature value corresponding to each element based on an amplitude value corresponding to each element in the target feature map and a preset amplitude threshold; determining a shape characteristic matrix composed of the shape characteristic values, performing norm operation on each column vector in the shape characteristic matrix, and determining first shape characteristic information corresponding to each column; and determining second shape characteristic information corresponding to each column based on the maximum shape characteristic value and the minimum shape characteristic value of each column in the shape characteristic matrix.
Based on the above aspects, the feature data determining module 330 includes:
the current characteristic diagram determining submodule is used for generating a current target characteristic diagram based on the second electromagnetic echo signal;
The current characteristic information determining submodule is used for extracting current amplitude information and current shape information in the current target characteristic diagram;
and the current characteristic data determining submodule is used for determining the current gesture characteristic data corresponding to the target object based on the current amplitude information and the current shape information.
Based on the above technical solutions, the current sleeping gesture determining module 340 includes:
a side lying type judging sub-module, configured to determine, based on the current side lying feature data, whether a current sleeping posture type corresponding to the target object is a side lying posture type;
and the pitching prone position judging sub-module is used for determining the current sleeping position category corresponding to the target object based on the current pitching lying characteristic data, the prone reference characteristic value and the supine reference characteristic value if the current sleeping position category corresponding to the target object is the non-lateral sleeping position category.
According to the technical scheme provided by the embodiment of the invention, the first electromagnetic echo signal received by the radar equipment in the first preset time period is obtained, wherein the radiation range of the radar equipment comprises the sleeping area of the target object, and the prone reference characteristic value and the supine reference characteristic value corresponding to the target object are determined based on the first electromagnetic echo signal, so that the personalized reference characteristic value of the target user can be obtained. When the gesture change of the target object after the first preset time period is detected, a second electromagnetic echo signal received by the radar equipment in the second preset time period is obtained, the current gesture characteristic data corresponding to the target object is determined based on the second electromagnetic echo signal, and then the current sleeping gesture category corresponding to the target object is determined based on the current gesture characteristic data, the prone reference characteristic value and the supine reference characteristic value. According to the method and the device, the technical problem that the accuracy rate of sleep gesture recognition based on the wearable sensor equipment is low is solved, a user does not need to wear various sensor equipment, foreign body sensation is not brought to the user, the non-contact recognition of the sleep gesture can be realized, the convenience of sleep gesture recognition is improved, and the accuracy rate of sleep gesture recognition is improved by acquiring the individualized sleep gesture feature reference value of the target object.
The sleep gesture recognition device provided by the embodiment of the disclosure can execute the sleep gesture recognition method provided by any embodiment of the disclosure, and has the corresponding functional modules and beneficial effects of the execution method.
It should be noted that each unit and module included in the above apparatus are only divided according to the functional logic, but not limited to the above division, so long as the corresponding functions can be implemented; in addition, the specific names of the functional units are also only for convenience of distinguishing from each other, and are not used to limit the protection scope of the embodiments of the present disclosure.
Example IV
Fig. 7 is a schematic structural diagram of an apparatus according to a fourth embodiment of the present invention. The device 10 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The device may also represent various forms of mobile apparatuses such as personal digital processing, cellular telephones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing apparatuses. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 7, the apparatus 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the device 10 can also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 13. An input/output (I/O) interface 15 is also connected to bus 13.
The various components in the device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a sleep gesture recognition method.
In some embodiments, the sleep gesture recognition method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the sleep gesture recognition method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the sleep gesture recognition method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable sleep gesture recognition apparatus, such that the computer programs, when executed by the processor, cause the functions/operations specified in the flowchart and/or block diagram to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome. It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein. The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A sleep posture recognition method, comprising:
acquiring a first electromagnetic echo signal received by radar equipment within a first preset time period; wherein the radiation range of the radar apparatus includes a sleep area of the target object;
determining a prone reference characteristic value and a supine reference characteristic value corresponding to the target object based on the first electromagnetic echo signal;
when the gesture change of the target object after the first preset time period is detected, acquiring a second electromagnetic echo signal received by the radar equipment within a second preset time period, and determining current gesture characteristic data corresponding to the target object based on the second electromagnetic echo signal;
and determining the current sleeping posture category corresponding to the target object based on the current posture characteristic data, the prone reference characteristic value and the supine reference characteristic value.
2. The sleep posture recognition method according to claim 1, wherein the determining, based on the first electromagnetic echo signal, a prone reference feature value and a supine reference feature value corresponding to the target object includes:
dividing the first electromagnetic echo signal into at least one electromagnetic echo signal segment, and determining gesture characteristic data of each electromagnetic echo signal segment; wherein the posture feature data includes lateral and pitch feature data;
For each electromagnetic echo signal segment, determining whether the sleep posture corresponding to the electromagnetic echo signal segment is a lateral posture or not based on the lateral characteristic data;
removing an electromagnetic echo signal segment with a sleeping posture being a lateral posture from the at least one electromagnetic echo signal segment to obtain a pitching horizontal echo signal segment;
and carrying out feature sorting on the pitching characteristic data corresponding to each pitching echo signal segment, and determining a prone reference characteristic value and a supine reference characteristic value corresponding to the target object.
3. The method of claim 2, wherein determining the gesture feature data for each electromagnetic echo signal segment comprises:
generating a target feature map based on electromagnetic echo signals for each of the electromagnetic echo signal segments;
extracting amplitude information and shape information in the target feature map;
and determining gesture characteristic data of the electromagnetic echo signal segment based on the amplitude information and the shape information.
4. A sleep gesture recognition method according to claim 3, wherein the amplitude information comprises: column vector magnitude information and row vector magnitude information, the extracting magnitude information in the target feature map includes:
Determining a column characteristic amplitude corresponding to each column vector based on amplitude values corresponding to elements in each column vector in the target characteristic map, and determining column vector amplitude information of the target characteristic map based on each column characteristic amplitude;
and determining the row characteristic amplitude corresponding to each row vector based on the amplitude value corresponding to each element in each row vector in the target characteristic diagram, and determining the row vector amplitude information of the target characteristic diagram based on each row characteristic amplitude.
5. A sleep gesture recognition method according to claim 3, wherein the shape information comprises: the extracting the shape information in the target feature map includes:
determining a shape characteristic value corresponding to each element based on the amplitude value corresponding to each element in the target characteristic diagram and a preset amplitude threshold;
determining a shape characteristic matrix composed of the shape characteristic values, performing norm operation on each column vector in the shape characteristic matrix, and determining first shape characteristic information corresponding to each column;
and determining second shape characteristic information corresponding to each column based on the maximum shape characteristic value and the minimum shape characteristic value of each column in the shape characteristic matrix.
6. The sleep gesture recognition method according to claim 1, wherein the determining current gesture feature data corresponding to the target object based on the second electromagnetic echo signal includes:
generating a current target feature map based on the second electromagnetic echo signal;
extracting current amplitude information and current shape information in the current target feature map;
and determining the current gesture characteristic data corresponding to the target object based on the current amplitude information and the current shape information.
7. The sleep posture recognition method of claim 1, wherein the current posture feature data includes current lateral and pitch-lateral feature data; the current sleeping gesture category includes: a lateral posture category, a supine posture category, and a prone posture category; the determining the current sleeping posture category corresponding to the target object based on the current posture feature data, the prone reference feature value and the supine reference feature value includes:
based on the current lateral lying characteristic data, determining whether the current sleeping posture category corresponding to the target object is a lateral lying posture category;
if not, determining the current sleeping posture category corresponding to the target object based on the current pitching feature data, the prone reference feature value and the supine reference feature value.
8. A sleep posture recognition apparatus, comprising:
the reference signal acquisition module is used for acquiring a first electromagnetic echo signal received by the radar equipment within a first preset time period; wherein the radiation range of the radar apparatus includes a sleep area of the target object;
the reference characteristic determining module is used for determining a prone reference characteristic value and a supine reference characteristic value corresponding to the target object based on the first electromagnetic echo signal;
the characteristic data determining module is used for acquiring a second electromagnetic echo signal received by the radar equipment within a second preset duration when the gesture change of the target object after the first preset duration is detected, and determining current gesture characteristic data corresponding to the target object based on the second electromagnetic echo signal;
and the current sleeping posture determining module is used for determining the current sleeping posture category corresponding to the target object based on the current posture characteristic data, the prone reference characteristic value and the supine reference characteristic value.
9. An electronic device, the electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the sleep gesture recognition method of any one of claims 1-7.
10. A computer readable storage medium storing computer instructions for causing a processor to perform the sleep gesture recognition method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310487224.9A CN116473547A (en) | 2023-05-04 | 2023-05-04 | Sleep gesture recognition method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310487224.9A CN116473547A (en) | 2023-05-04 | 2023-05-04 | Sleep gesture recognition method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116473547A true CN116473547A (en) | 2023-07-25 |
Family
ID=87219307
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310487224.9A Pending CN116473547A (en) | 2023-05-04 | 2023-05-04 | Sleep gesture recognition method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116473547A (en) |
-
2023
- 2023-05-04 CN CN202310487224.9A patent/CN116473547A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110458061B (en) | Method for identifying old people falling down and accompanying robot | |
| CN107169454B (en) | Face image age estimation method and device and terminal equipment thereof | |
| WO2021218753A1 (en) | Gesture recognition method and related apparatus | |
| WO2019200782A1 (en) | Sample data classification method, model training method, electronic device and storage medium | |
| KR20170091963A (en) | Gesture classification apparatus and method using electromyogram signals | |
| CN111954250B (en) | Lightweight Wi-Fi behavior sensing method and system | |
| Kalsh et al. | Sign language recognition system | |
| US20180177415A1 (en) | Cardiovascular disease detection | |
| CN111149104A (en) | Apparatus, method and computer program product for biometric identification | |
| Zhang et al. | Human activity recognition across scenes and categories based on CSI | |
| CN112801000A (en) | Household old man falling detection method and system based on multi-feature fusion | |
| CN111053552B (en) | QRS wave detection method based on deep learning | |
| Qiao et al. | Human activity classification based on micro-Doppler signatures separation | |
| CN114529945A (en) | Emotion recognition method, device, equipment and storage medium | |
| CN108898062B (en) | Hand motion recognition method based on improved signal segment extraction algorithm | |
| CN116482680B (en) | Body interference identification method, device, system and storage medium | |
| CN116473547A (en) | Sleep gesture recognition method and device, electronic equipment and storage medium | |
| CN109063652B (en) | Signal processing method, system and computer storage medium | |
| CN111913575A (en) | Method for recognizing hand-language words | |
| CN111407261A (en) | Method and device for measuring periodic information of biological signal and electronic equipment | |
| CN114764580A (en) | Real-time human body gesture recognition method based on no-wearing equipment | |
| CN113208566A (en) | Data processing method and device, electronic equipment and storage medium | |
| Qin et al. | Consumer Wi‐Fi device based action quality recognition: An illustrative example of seated dumbbell press action | |
| Liu et al. | A novel denoising method for infrared image based on bilateral filtering and non-local means | |
| CN112861679A (en) | Transfer learning method and system for behavior recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |