CN116311539B - Sleep motion capturing method, device, equipment and storage medium based on millimeter waves - Google Patents
Sleep motion capturing method, device, equipment and storage medium based on millimeter waves Download PDFInfo
- Publication number
- CN116311539B CN116311539B CN202310570275.8A CN202310570275A CN116311539B CN 116311539 B CN116311539 B CN 116311539B CN 202310570275 A CN202310570275 A CN 202310570275A CN 116311539 B CN116311539 B CN 116311539B
- Authority
- CN
- China
- Prior art keywords
- motion
- action
- image
- sleep
- monitoring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/62—Extraction of image or video features relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The invention relates to an intelligent monitoring technology and discloses a sleep motion capturing method, device, equipment and storage medium based on millimeter waves. The method comprises the following steps: monitoring a real-time monitoring image of a monitoring area shot by the millimeter wave radar; when the object variation exists in the monitoring area, intercepting an action image sequence in the real-time monitoring image; performing motion recognition on the motion image sequence by using a pre-trained sleep motion monitoring model to obtain a motion recognition result, and performing an independent classification operation on the image feature sequences in the motion image sequence according to the motion recognition result to obtain a motion independent feature set; identifying the independent feature set of the motion to obtain a micro motion set during motion; and when no object change exists in the monitoring area, a short-time Fourier transform algorithm is utilized to identify a static minute action set in the real-time monitoring image. The invention can improve the accuracy of capturing the micro-motions and the sleep monitoring quality.
Description
Technical Field
The present invention relates to the field of intelligent monitoring technologies, and in particular, to a millimeter wave-based sleep motion capturing method, apparatus, device, and computer-readable storage medium.
Background
With the development of health monitoring technology, more and more health monitoring products, such as intelligent monitoring products for sleep monitoring, sports monitoring, and senior citizen monitoring, are appeared.
The millimeter-level microwave radar has the advantages of both the microwave radar and the photoelectric radar, so that most of today intelligent monitoring products perform motion capture through the millimeter-level microwave radar, for example, some intelligent monitoring products can identify whether the motion of a human body reaches the standard during motion, but in the sleep monitoring process, the motion amplitude of the human body is slight, and when micro motions such as muscle twitches, eye movements, breath holding and the like occur, the condition of insufficient sensitivity is easy to occur, and the sleep monitoring result is inaccurate.
Disclosure of Invention
The invention provides a millimeter wave-based sleep motion capturing method, device, equipment and storage medium, which mainly aim to improve accuracy of capturing micro motions and improve sleep monitoring quality.
In order to achieve the above object, the present invention provides a sleep motion capturing method based on millimeter waves, including:
Acquiring a real-time monitoring image of a monitoring area shot by a millimeter wave radar, and judging whether an object change exists in the monitoring area according to a preset change amplitude threshold;
when the object variation exists in the monitoring area, intercepting and obtaining an action image sequence from the real-time monitoring image;
performing motion recognition on the motion image sequence by using a pre-trained sleep motion monitoring model to obtain a motion recognition result, and performing an independent classification operation on the image feature sequences in the motion image sequence according to the motion recognition result to obtain a motion independent feature set;
performing self-adaptive weight increase on the motion independent feature set, and performing motion recognition operation on a weight increase result to obtain a motion time micro motion set;
when no object change exists in the monitoring area, performing signal conversion on the real-time monitoring image to obtain a frequency domain signal, and performing motion monitoring on the frequency domain signal based on signal phase information by utilizing a short-time Fourier transform algorithm to obtain a static time micro motion set;
and recording the static micro-action set, the action recognition result and the dynamic micro-action set according to the time sequence of the real-time monitoring image to obtain a sleep action monitoring result.
Optionally, before the action recognition is performed on the action image sequence by using the pre-trained sleep action monitoring model, the method further includes:
performing image feature extraction and identification on the pre-constructed millimeter wave sleep image to obtain an image change feature set, and performing action identification on the image change feature set to obtain an action category;
classifying the image fluctuation feature set according to a preset weight threshold and the weight of each image fluctuation feature in the image fluctuation feature set when the action category is obtained, obtaining a micro feature set, and carrying out micro action recognition on the micro feature set to obtain a micro action category;
constructing key value pairs by utilizing the millimeter wave sleep image, the action category and the micro action category to obtain a training sample;
forward network prediction is carried out on the training sample by utilizing a pre-constructed sleep action monitoring model, so as to obtain an action prediction result and a micro action prediction result;
calculating auxiliary loss between the action category of the training sample and the action prediction result and calculating output loss between the micro action category of the training sample and the micro action prediction result by using a cross entropy loss algorithm;
And according to the gradient descent method, the auxiliary loss and the output loss, carrying out network parameter reverse updating on the sleep action monitoring model to obtain the trained sleep action monitoring model.
Optionally, the classifying operation based on independence is performed on the image feature sequences in the motion image sequence according to the motion recognition result to obtain a motion independent feature set, which includes:
when the action recognition result is obtained and output, the weight of each image feature in the action image sequence is obtained, and the image feature with the weight larger than a preset first threshold value is obtained, so that an action association feature set is obtained;
according to the attention weight configuration rule, calculating the attention weight of each action associated feature in the action associated feature set for the action recognition result, and extracting action associated features with attention weight greater than a preset second threshold value to obtain an action key feature set;
and deleting the image features related to the motion key feature set in each image feature in the motion image sequence to obtain a motion independent feature set.
Optionally, the determining whether the object variation exists in the monitoring area according to the preset variation amplitude threshold includes:
Performing color gamut cutting operation on the real-time monitoring image to obtain each gray scale block, performing clustering operation on each gray scale block, and calculating the area size and the center point of each clustered gray scale block;
calculating the relative change of the area size of each gray scale block in a plurality of real-time monitoring images in a preset time period to obtain a first change value;
calculating the relative displacement of the center point of each gray scale block in the plurality of real-time monitoring images in the preset time period according to a preset coordinate system to obtain a second change value;
and carrying out weighted summation on the first change value and the second change value according to a preset weight configuration coefficient to obtain a comprehensive change value, and judging whether an object change exists in the monitoring area according to whether the comprehensive change value is larger than a preset change amplitude threshold value.
Optionally, the performing the motion recognition on the motion image sequence by using the pre-trained sleep motion monitoring model to obtain a motion recognition result includes:
performing convolution operation on the action image sequence to obtain a convolution matrix set;
carrying out average pooling operation on the convolution matrix to obtain a pooling matrix set;
Flattening each pooling matrix in the pooling matrix set to obtain each characteristic sequence;
and splicing the characteristic sequences, and performing full-connection classification operation on the spliced results to obtain an action recognition result.
Optionally, the sleep action monitoring model comprises a feature extraction network, an action recognition network and an independence characteristic classification network.
Optionally, after the sleep action monitoring result is obtained, the method further includes:
storing the sleep action monitoring result into a pre-constructed data summarization database, and extracting features of the data summarization database by utilizing a pre-constructed sleep quality assessment model to obtain a sleep feature sequence;
and carrying out sleep quality assessment on the sleep characteristic sequence to obtain a sleep assessment result.
In order to solve the above problems, the present invention also provides a sleep motion capture device based on millimeter waves, the device comprising:
the scene classification module is used for acquiring real-time monitoring images of a monitoring area shot by the millimeter wave radar and judging whether an object change exists in the monitoring area according to a preset change amplitude threshold value;
the motion time motion recognition module is used for intercepting a motion image sequence from the real-time monitoring image when the object variation exists in the monitoring area, performing motion recognition on the motion image sequence by utilizing a pre-trained sleep motion monitoring model to obtain a motion recognition result, performing an independent classification operation on an image feature sequence in the motion image sequence according to the motion recognition result to obtain a motion independent feature set, performing adaptive weight increase on the motion independent feature set, and performing motion recognition operation on a weight increase result to obtain a motion time micro motion set;
The static time action recognition module is used for carrying out signal conversion on the real-time monitoring image to obtain a frequency domain signal when no object fluctuation exists in the monitoring area, and carrying out action monitoring on the frequency domain signal based on signal phase information by utilizing a short-time Fourier transform algorithm to obtain a static time micro action set;
and the action recording module is used for recording the static time micro action set, the action recognition result and the dynamic time micro action set according to the time sequence of the real-time monitoring image to obtain a sleep action monitoring result.
In order to solve the above-mentioned problems, the present invention also provides an electronic apparatus including:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the millimeter wave based sleep motion capture method described above.
In order to solve the above-mentioned problems, the present invention also provides a computer-readable storage medium having stored therein at least one computer program that is executed by a processor in an electronic device to implement the above-mentioned millimeter wave based sleep motion capture method.
When no object change exists in the monitoring area, the embodiment of the invention firstly carries out micro-action recognition in a mode of carrying out signal conversion on the monitoring image to obtain a static time micro-action set; when the object variation exists in the monitoring area, on one hand, large motion recognition is performed on the motion image sequence through motion recognition, and on the other hand, the feature mutually independent of the motion recognition result is extracted from the image feature sequence of the motion image sequence, so that a motion independent feature set is obtained; further, for the motion independent feature set, identifying a micro motion when a large motion occurs, and obtaining a motion micro motion set; and finally, obtaining an accurate and complete sleep action monitoring result. Therefore, the method, the device, the equipment and the storage medium for capturing the sleep motion based on the millimeter wave can improve accuracy of capturing the micro motion and improve sleep monitoring quality.
Drawings
Fig. 1 is a schematic flow chart of a sleep motion capturing method based on millimeter waves according to an embodiment of the present invention;
FIG. 2 is a detailed flowchart of one step in a millimeter wave based sleep motion capture method according to an embodiment of the present invention;
FIG. 3 is a detailed flowchart of one step in a millimeter wave based sleep motion capture method according to an embodiment of the present invention;
FIG. 4 is a detailed flowchart of one step in a millimeter wave based sleep motion capture method according to an embodiment of the present invention;
FIG. 5 is a functional block diagram of a millimeter wave based sleep motion capture device according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an electronic device implementing the sleep motion capturing method based on millimeter waves according to an embodiment of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The embodiment of the application provides a sleep motion capturing method based on millimeter waves. In the embodiment of the present application, the execution body of the sleep motion capture method based on millimeter waves includes, but is not limited to, at least one of a server, a terminal, and the like, which can be configured to execute the method provided in the embodiment of the present application. In other words, the millimeter wave based sleep motion capture method may be performed by software or hardware installed in a terminal device or a server device, and the software may be a blockchain platform. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (ContentDelivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.
Referring to fig. 1, a schematic flow chart of a sleep motion capturing method based on millimeter waves according to an embodiment of the invention is shown. In this embodiment, the millimeter wave-based sleep motion capture method includes:
s1, acquiring a real-time monitoring image of a monitoring area shot by the millimeter wave radar.
In the embodiment of the invention, the real-time monitoring image shot by the millimeter wave radar is a gray level image, and the image brightness in the real-time monitoring image represents the reflectivity or scattering intensity of the scanned object.
S2, judging whether an object change exists in the monitoring area according to a preset change amplitude threshold value.
In detail, referring to fig. 2, in the embodiment of the present invention, the determining whether there is an object variation in the monitoring area according to the preset variation amplitude threshold includes:
s21, performing color gamut cutting operation on the real-time monitoring image to obtain each gray scale block, performing clustering operation on each gray scale block, and calculating the area size and the center point of each clustered gray scale block;
s22, calculating the relative change of the area size of each gray scale block in a plurality of real-time monitoring images in a preset time period to obtain a first change value;
S23, calculating the relative displacement of the center point of each gray scale block in the plurality of real-time monitoring images in the preset time period according to a preset coordinate system to obtain a second change value;
and S24, carrying out weighted summation on the first change value and the second change value according to a preset weight configuration coefficient to obtain a comprehensive change value, and judging whether an object change exists in the monitoring area according to whether the comprehensive change value is larger than a preset change amplitude threshold value.
The color gamut cutting method is a method for dividing according to different gray values of different parts in an image. The embodiment of the invention controls the change of the area size of each gray scale block and the clustering center point by identifying the area size of each gray scale block, wherein the center point is used for judging fluctuation, and the area size can judge whether the target object changes the gesture. According to the embodiment of the invention, the weight configuration coefficient and the variation amplitude threshold which accord with the household sleep monitoring scene are obtained according to the simulation experiment, and the action of judging whether the object variation exists in the monitoring area is performed.
And S3, when the real-time monitoring image has object variation, intercepting and obtaining an action image sequence from the real-time monitoring image.
S4, performing action recognition on the action image sequence by using a pre-trained sleep action monitoring model to obtain an action recognition result, and performing independent classification operation on the image feature sequences in the action image sequence according to the action recognition result to obtain an action independent feature set.
In the embodiment of the invention, the sleep action monitoring model comprises a feature extraction network, an action recognition network and an independence characteristic classification network. The characteristic extraction network is used for identifying image information in the real-time monitoring image, the action identification network can identify normal human actions and micro actions, and the independent characteristic classification network is used for extracting characteristics irrelevant to larger actions and adaptively increasing the weight of the characteristics.
It should be noted that when a large-amplitude motion occurs, a small sleep motion is easily disturbed, for example, when a person turns over, a large disturbance is caused to the recognition of features such as respiration, eye movement, etc., so that it is necessary to preferentially recognize a large motion, obtain a motion recognition result, and then re-recognize image features to which the motion is not directly related. In addition, since there is coordination between the human body movements, most of the learned characteristics may be coordinated movements in the movement recognition process, for example, when the body is turned, the movement of the body tissue may be a corresponding direct characteristic, and the respiratory exacerbation, muscle contraction, etc. may be related characteristics, while the blood pressure elevation, eye movement, etc. may be unrelated characteristics.
In detail, in the embodiment of the present invention, the performing, by using the pre-trained sleep action monitoring model, action recognition on the action image sequence to obtain an action recognition result includes: performing convolution operation on the action image sequence to obtain a convolution matrix set; carrying out average pooling operation on the convolution matrix to obtain a pooling matrix set; flattening each pooling matrix in the pooling matrix set to obtain each characteristic sequence; and splicing the characteristic sequences, and performing full-connection classification operation on the spliced results to obtain an action recognition result.
The embodiment of the invention utilizes the feature extraction network in the sleep action monitoring model to carry out convolution, pooling and flattening operations on the action image sequence, carries out feature extraction, and then carries out full connection operation through the action recognition network to obtain an action recognition result. The convolution is used for extracting the features in the action image sequence, the pooling and flattening are used for performing dimension reduction operation on the extracted features, the calculation difficulty can be reduced under the condition that the feature quantity is not affected, and the full connection operation refers to the operation of randomly combining the features and performing classification recognition.
Further, referring to fig. 3, in the embodiment of the present invention, according to the motion recognition result, performing an operation of classifying the image feature sequences in the motion image sequence based on independence to obtain a motion independent feature set, including:
s41, when the action recognition result is obtained and output, the weight of each image feature in the action image sequence is obtained, and the image feature with the weight larger than a preset first threshold value is obtained, so that an action association feature set is obtained;
s42, according to an attention weight configuration rule, calculating the attention weight of each action associated feature in the action associated feature set for the action recognition result, and extracting action associated features with attention weights greater than a preset second threshold value to obtain an action key feature set;
s43, deleting the image features related to the motion key feature set in each image feature in the motion image sequence to obtain a motion independent feature set.
In the embodiment of the invention, the weight of each image characteristic when the action recognition result is output is obtained by inquiring the action recognition network, and then the action association characteristic set is obtained by comparing a preset first threshold value, for example, 0.3. However, since there are many related actions in the action-related feature set, the related actions in the action-related feature set need to be separated from the key actions.
In the embodiment of the invention, through a dot product attention weight configuration mechanism, the attention weight between each action associated feature and the action recognition result is recognized, and the action associated feature with the attention weight greater than a preset second threshold value, for example, 0.4 is defined as the action key feature, so as to obtain the action key feature set.
According to the embodiment of the invention, the motion independent feature set is obtained by deleting the image features related to the motion key feature set in each image feature in the motion image sequence, and the motion independent feature set contains all the features of the micro motions interfered by the large motions.
Further, referring to fig. 4, in an embodiment of the present invention, before the pre-trained sleep motion monitoring model is used to perform motion recognition on the motion image sequence, the method further includes:
s401, performing image feature extraction and identification on a pre-constructed millimeter wave sleep image to obtain an image variation feature set, and performing action identification on the image variation feature set to obtain an action category;
s402, classifying the image change feature set according to a preset weight threshold and the weight of each image change feature in the image change feature set when the action category is obtained, so as to obtain a micro feature set, and carrying out micro action recognition on the micro feature set so as to obtain a micro action category;
S403, constructing key value pairs by utilizing the millimeter wave sleep image, the action category and the micro action category to obtain a training sample;
s404, forward network prediction is carried out on the training sample by utilizing a pre-constructed sleep action monitoring model, so as to obtain an action prediction result and a micro action prediction result;
s405, calculating auxiliary loss between the action category of the training sample and the action prediction result by using a cross entropy loss algorithm, and calculating output loss between the micro action category of the training sample and the micro action prediction result;
s406, according to the gradient descent method, the auxiliary loss and the output loss, carrying out network parameter reverse updating on the sleep action monitoring model to obtain a trained sleep action monitoring model.
According to the embodiment of the invention, the motion type can be obtained by performing motion recognition through a model or algorithm specially recognizing the conventional motion, then the weight of the micro feature set is increased in a weight increasing mode, the micro motion is recognized by using the model or algorithm specially recognizing the micro motion, the micro motion type is obtained, and then the training sample of the millimeter wave sleep image is constructed through the motion type and the micro motion type and then participates in the training process.
The first output of the action recognition network of the sleep action monitoring model is set as an auxiliary task through an auxiliary loss (auxiliary loss) method, the second output of the action recognition network is used as an output result, namely, the recognition result of the conventional action is used as an intermediate link, the loss is calculated independently, and the loss of the tiny action recognition is used as a model loss. Therefore, the conventional action recognition process of the action recognition network is only carried out in the computer training process, and does not occur in the training progress display process, and only the output result of the second output of the action recognition network is displayed in the training progress display process. The added auxiliary loss in the training process can accelerate convergence, improve model training efficiency, enhance supervision and enhance reverse propagation of gradients, so that the sleep action monitoring model can perform machine learning of two actions more efficiently.
The cross entropy loss algorithm and the gradient descent method are adopted in the training process. The cross entropy loss algorithm is used for monitoring errors between the action category of the training sample and the action prediction result and errors between the micro action category of the training sample and the micro action prediction result, and the gradient descent method is a method for calculating a minimum value of a function and is used for updating model network parameters through loss values.
S5, performing self-adaptive weight increase on the motion independent feature set, and performing motion recognition operation on a weight increase result to obtain a motion time micro motion set.
In the embodiment of the invention, the action recognition network can recognize both normal actions and micro actions, so that the weights of the independent feature sets of the actions are required to be corresponding to the image features corresponding to the normal actions. According to the embodiment of the invention, the weight magnitude is automatically recorded through the preset interface buried points, the weight of the action independent feature set is ensured, and the action micro action set is accurately identified.
When no object change exists in the monitoring area, S6, performing signal conversion on the real-time monitoring image to obtain a frequency domain signal, and performing motion monitoring on the frequency domain signal based on signal phase information by utilizing a short-time Fourier transform algorithm to obtain a static time micro motion set;
according to the embodiment of the invention, noise reduction, filtering and enhancement operations are carried out on the real-time monitoring image according to the preprocessing configuration coefficient conforming to the sleep monitoring scene, the image quality of the real-time monitoring image is improved, then the real-time monitoring image is sampled according to the sampling rate in the preprocessing configuration coefficient to obtain a sample point sequence set, and then the real-time monitoring image is converted into a frequency domain signal through discrete Fourier change. Then, the embodiment of the invention decomposes the signal into a narrow-band frequency component set through short-time Fourier change, and further monitors the micro-motion of the human body by utilizing the phase information of each frequency component to obtain a static micro-motion set. The preprocessing configuration coefficients comprise sampling rate, sampling depth and the like which accord with a scene, and influence on a real-time monitoring image is avoided.
And S7, recording the static micro-action set, the action recognition result and the dynamic micro-action set according to the time sequence of the real-time monitoring image to obtain a sleep action monitoring result.
The embodiment of the invention records the static micro-action set, the action recognition result and the dynamic micro-action set, so that the recognition of the micro-action is not influenced even when a large action occurs, and an accurate and complete sleep action monitoring result is finally obtained.
In addition, in another embodiment of the present invention, after the sleep action monitoring result is obtained, the method further includes:
storing the sleep action monitoring result into a pre-constructed data summarization database, and extracting features of the data summarization database by utilizing a pre-constructed sleep quality assessment model to obtain a sleep feature sequence;
and carrying out sleep quality assessment on the sleep characteristic sequence to obtain a sleep assessment result.
In the embodiment of the invention, the data summarizing database can integrate the data acquired by equipment or models other than the sleep action monitoring model, such as a health monitoring watch, an infrared temperature monitor and the like, through a plurality of data interfaces to obtain all-aspect monitoring data, and then perform overall data analysis through a pre-constructed sleep quality assessment model to obtain an accurate sleep assessment result.
When no object change exists in the monitoring area, the embodiment of the invention firstly carries out micro-action recognition in a mode of carrying out signal conversion on the monitoring image to obtain a static time micro-action set; when the object variation exists in the monitoring area, on one hand, large motion recognition is performed on the motion image sequence through motion recognition, and on the other hand, the feature mutually independent of the motion recognition result is extracted from the image feature sequence of the motion image sequence, so that a motion independent feature set is obtained; further, for the motion independent feature set, identifying a micro motion when a large motion occurs, and obtaining a motion micro motion set; and finally, obtaining an accurate and complete sleep action monitoring result. Therefore, the millimeter wave-based sleep motion capturing method provided by the embodiment of the invention can improve the accuracy of capturing micro motions and improve the sleep monitoring quality.
Fig. 5 is a functional block diagram of a millimeter wave-based sleep motion capture device according to an embodiment of the present invention.
The millimeter wave-based sleep motion capture device 100 of the present invention may be installed in an electronic device. Depending on the functions implemented, the millimeter wave based sleep motion capture device 100 may include a scene classification module 101, a motion-by-motion recognition module 102, a static-by-motion recognition module 103, and a motion recording module 104. The module of the invention, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the electronic device, capable of being executed by the processor of the electronic device and of performing a fixed function.
In the present embodiment, the functions concerning the respective modules/units are as follows:
the scene classification module 101 is configured to obtain a real-time monitoring image of a monitoring area captured by a millimeter wave radar, and determine whether an object variation exists in the monitoring area according to a preset variation amplitude threshold;
the motion-time motion recognition module 102 is configured to intercept the motion image sequence from the real-time monitoring image when there is an object variation in the monitoring area, perform motion recognition on the motion image sequence by using a pre-trained sleep motion monitoring model to obtain a motion recognition result, perform an independence-based classification operation on an image feature sequence in the motion image sequence according to the motion recognition result to obtain a motion independent feature set, perform adaptive weight increase on the motion independent feature set, and perform motion recognition operation on a weight increase result to obtain a motion-time micro motion set;
the static time action recognition module 103 is configured to perform signal conversion on the real-time monitoring image to obtain a frequency domain signal when there is no object variation in the monitoring area, and perform action monitoring on the frequency domain signal based on signal phase information by using a short-time fourier transform algorithm to obtain a static time micro action set;
The action recording module 104 is configured to record the static micro-action set, the action recognition result, and the dynamic micro-action set according to the time sequence of the real-time monitoring image, so as to obtain a sleep action monitoring result.
In detail, each module in the millimeter wave-based sleep motion capture device 100 in the embodiment of the present application adopts the same technical means as the millimeter wave-based sleep motion capture method described in fig. 1 to 4 and can produce the same technical effects when in use, and will not be described here again.
Fig. 6 is a schematic structural diagram of an electronic device 1 implementing a sleep motion capture method based on millimeter waves according to an embodiment of the present invention.
The electronic device 1 may comprise a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may further comprise a computer program, such as a millimeter wave based sleep motion capture program, stored in the memory 11 and executable on the processor 10.
The processor 10 may be formed by an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be formed by a plurality of integrated circuits packaged with the same function or different functions, including one or more central processing units (Central Processing Unit, CPU), a microprocessor, a digital processing chip, a graphics processor, a combination of various control chips, and so on. The processor 10 is a Control Unit (Control Unit) of the electronic device 1, connects various components of the entire electronic device using various interfaces and lines, executes programs or modules stored in the memory 11 (for example, executes a millimeter wave-based sleep motion capture program, etc.), and invokes data stored in the memory 11 to perform various functions of the electronic device and process data.
The memory 11 includes at least one type of readable storage medium including flash memory, a removable hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device, such as a mobile hard disk of the electronic device. The memory 11 may in other embodiments also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the electronic device. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 may be used not only to store application software installed in an electronic device and various types of data, such as codes of a sleep motion capture program based on millimeter waves, but also to temporarily store data that has been output or is to be output.
The communication bus 12 may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. The bus is arranged to enable a connection communication between the memory 11 and at least one processor 10 etc.
The communication interface 13 is used for communication between the electronic device 1 and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), typically used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a Display (Display), an input unit such as a Keyboard (Keyboard), or alternatively a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device and for displaying a visual user interface.
Fig. 6 shows only an electronic device with components, it being understood by a person skilled in the art that the structure shown in fig. 6 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or may combine certain components, or may be arranged in different components.
For example, although not shown, the electronic device 1 may further include a power source (such as a battery) for supplying power to each component, and preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that functions of charge management, discharge management, power consumption management, and the like are implemented through the power management device. The power supply may also include one or more of any of a direct current or alternating current power supply, recharging device, power failure detection circuit, power converter or inverter, power status indicator, etc. The electronic device 1 may further include various sensors, bluetooth modules, wi-Fi modules, etc., which will not be described herein.
It should be understood that the embodiments described are for illustrative purposes only and are not limited to this configuration in the scope of the patent application.
The millimeter wave based sleep motion capture program stored by the memory 11 in the electronic device 1 is a combination of instructions that, when executed in the processor 10, may implement:
acquiring a real-time monitoring image of a monitoring area shot by a millimeter wave radar, and judging whether an object change exists in the monitoring area according to a preset change amplitude threshold;
When the object variation exists in the monitoring area, intercepting and obtaining an action image sequence from the real-time monitoring image;
performing motion recognition on the motion image sequence by using a pre-trained sleep motion monitoring model to obtain a motion recognition result, and performing an independent classification operation on the image feature sequences in the motion image sequence according to the motion recognition result to obtain a motion independent feature set;
performing self-adaptive weight increase on the motion independent feature set, and performing motion recognition operation on a weight increase result to obtain a motion time micro motion set;
when no object change exists in the monitoring area, performing signal conversion on the real-time monitoring image to obtain a frequency domain signal, and performing motion monitoring on the frequency domain signal based on signal phase information by utilizing a short-time Fourier transform algorithm to obtain a static time micro motion set;
and recording the static micro-action set, the action recognition result and the dynamic micro-action set according to the time sequence of the real-time monitoring image to obtain a sleep action monitoring result.
In particular, the specific implementation method of the above instructions by the processor 10 may refer to the description of the relevant steps in the corresponding embodiment of the drawings, which is not repeated herein.
Further, the modules/units integrated in the electronic device 1 may be stored in a computer readable storage medium if implemented in the form of software functional units and sold or used as separate products. The computer readable storage medium may be volatile or nonvolatile. For example, the computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM).
The present invention also provides a computer readable storage medium storing a computer program which, when executed by a processor of an electronic device, can implement:
acquiring a real-time monitoring image of a monitoring area shot by a millimeter wave radar, and judging whether an object change exists in the monitoring area according to a preset change amplitude threshold;
when the object variation exists in the monitoring area, intercepting and obtaining an action image sequence from the real-time monitoring image;
performing motion recognition on the motion image sequence by using a pre-trained sleep motion monitoring model to obtain a motion recognition result, and performing an independent classification operation on the image feature sequences in the motion image sequence according to the motion recognition result to obtain a motion independent feature set;
Performing self-adaptive weight increase on the motion independent feature set, and performing motion recognition operation on a weight increase result to obtain a motion time micro motion set;
when no object change exists in the monitoring area, performing signal conversion on the real-time monitoring image to obtain a frequency domain signal, and performing motion monitoring on the frequency domain signal based on signal phase information by utilizing a short-time Fourier transform algorithm to obtain a static time micro motion set;
and recording the static micro-action set, the action recognition result and the dynamic micro-action set according to the time sequence of the real-time monitoring image to obtain a sleep action monitoring result.
In the several embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be other manners of division when actually implemented.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.
The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.
The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.
Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or means recited in the system claims can also be implemented by means of software or hardware by means of one unit or means. The terms first, second, etc. are used to denote a name, but not any particular order.
Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.
Claims (10)
1. A millimeter wave-based sleep motion capture method, the method comprising:
Acquiring a real-time monitoring image of a monitoring area shot by a millimeter wave radar, and judging whether an object change exists in the monitoring area according to a preset change amplitude threshold;
when the object variation exists in the monitoring area, intercepting and obtaining an action image sequence from the real-time monitoring image;
performing motion recognition on the motion image sequence by using a pre-trained sleep motion monitoring model to obtain a motion recognition result, and performing an independent classification operation on the image feature sequences in the motion image sequence according to the motion recognition result to obtain a motion independent feature set;
performing self-adaptive weight increase on the motion independent feature set, and performing motion recognition operation on a weight increase result to obtain a motion time micro motion set;
when no object change exists in the monitoring area, performing signal conversion on the real-time monitoring image to obtain a frequency domain signal, and performing motion monitoring on the frequency domain signal based on signal phase information by utilizing a short-time Fourier transform algorithm to obtain a static time micro motion set;
and recording the static micro-action set, the action recognition result and the dynamic micro-action set according to the time sequence of the real-time monitoring image to obtain a sleep action monitoring result.
2. The millimeter wave based sleep motion capture method of claim 1, wherein prior to said motion recognition of said sequence of motion images using a pre-trained sleep motion monitoring model, said method further comprises:
performing image feature extraction and identification on the pre-constructed millimeter wave sleep image to obtain an image change feature set, and performing action identification on the image change feature set to obtain an action category;
classifying the image fluctuation feature set according to a preset weight threshold and the weight of each image fluctuation feature in the image fluctuation feature set when the action category is obtained, obtaining a micro feature set, and carrying out micro action recognition on the micro feature set to obtain a micro action category;
constructing key value pairs by utilizing the millimeter wave sleep image, the action category and the micro action category to obtain a training sample;
forward network prediction is carried out on the training sample by utilizing a pre-constructed sleep action monitoring model, so as to obtain an action prediction result and a micro action prediction result;
calculating auxiliary loss between the action category of the training sample and the action prediction result and calculating output loss between the micro action category of the training sample and the micro action prediction result by using a cross entropy loss algorithm;
And according to the gradient descent method, the auxiliary loss and the output loss, carrying out network parameter reverse updating on the sleep action monitoring model to obtain the trained sleep action monitoring model.
3. The millimeter wave-based sleep motion capture method of claim 1, wherein the performing an independence-based classification operation on the image feature sequences in the motion image sequence according to the motion recognition result to obtain a motion independent feature set comprises:
when the action recognition result is obtained and output, the weight of each image feature in the action image sequence is obtained, and the image feature with the weight larger than a preset first threshold value is obtained, so that an action association feature set is obtained;
according to the attention weight configuration rule, calculating the attention weight of each action associated feature in the action associated feature set for the action recognition result, and extracting action associated features with attention weight greater than a preset second threshold value to obtain an action key feature set;
and deleting the image features related to the motion key feature set in each image feature in the motion image sequence to obtain a motion independent feature set.
4. The millimeter wave-based sleep motion capture method of claim 1, wherein the determining whether there is an object change in the monitored area according to a preset change amplitude threshold value comprises:
performing color gamut cutting operation on the real-time monitoring image to obtain each gray scale block, performing clustering operation on each gray scale block, and calculating the area size and the center point of each clustered gray scale block;
calculating the relative change of the area size of each gray scale block in a plurality of real-time monitoring images in a preset time period to obtain a first change value;
calculating the relative displacement of the center point of each gray scale block in the plurality of real-time monitoring images in the preset time period according to a preset coordinate system to obtain a second change value;
and carrying out weighted summation on the first change value and the second change value according to a preset weight configuration coefficient to obtain a comprehensive change value, and judging whether an object change exists in the monitoring area according to whether the comprehensive change value is larger than a preset change amplitude threshold value.
5. The millimeter wave-based sleep motion capture method of claim 1, wherein the performing motion recognition on the sequence of motion images using a pre-trained sleep motion monitoring model to obtain a motion recognition result comprises:
Performing convolution operation on the action image sequence to obtain a convolution matrix set;
carrying out average pooling operation on the convolution matrix to obtain a pooling matrix set;
flattening each pooling matrix in the pooling matrix set to obtain each characteristic sequence;
and splicing the characteristic sequences, and performing full-connection classification operation on the spliced results to obtain an action recognition result.
6. The millimeter wave based sleep motion capture method of claim 1, wherein the sleep motion monitoring model comprises a feature extraction network, a motion recognition network, an independence feature classification network.
7. The millimeter wave based sleep motion capture method of claim 1, wherein after said deriving sleep motion monitoring results, the method further comprises:
storing the sleep action monitoring result into a pre-constructed data summarization database, and extracting features of the data summarization database by utilizing a pre-constructed sleep quality assessment model to obtain a sleep feature sequence;
and carrying out sleep quality assessment on the sleep characteristic sequence to obtain a sleep assessment result.
8. A millimeter wave based sleep motion capture device, the device comprising:
The scene classification module is used for acquiring real-time monitoring images of a monitoring area shot by the millimeter wave radar and judging whether an object change exists in the monitoring area according to a preset change amplitude threshold value;
the motion time motion recognition module is used for intercepting a motion image sequence from the real-time monitoring image when the object variation exists in the monitoring area, performing motion recognition on the motion image sequence by utilizing a pre-trained sleep motion monitoring model to obtain a motion recognition result, performing an independent classification operation on an image feature sequence in the motion image sequence according to the motion recognition result to obtain a motion independent feature set, performing adaptive weight increase on the motion independent feature set, and performing motion recognition operation on a weight increase result to obtain a motion time micro motion set;
the static time action recognition module is used for carrying out signal conversion on the real-time monitoring image to obtain a frequency domain signal when no object fluctuation exists in the monitoring area, and carrying out action monitoring on the frequency domain signal based on signal phase information by utilizing a short-time Fourier transform algorithm to obtain a static time micro action set;
And the action recording module is used for recording the static time micro action set, the action recognition result and the dynamic time micro action set according to the time sequence of the real-time monitoring image to obtain a sleep action monitoring result.
9. An electronic device, the electronic device comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the millimeter wave based sleep motion capture method of any one of claims 1 to 7.
10. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the millimeter wave based sleep motion capture method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310570275.8A CN116311539B (en) | 2023-05-19 | 2023-05-19 | Sleep motion capturing method, device, equipment and storage medium based on millimeter waves |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310570275.8A CN116311539B (en) | 2023-05-19 | 2023-05-19 | Sleep motion capturing method, device, equipment and storage medium based on millimeter waves |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116311539A CN116311539A (en) | 2023-06-23 |
| CN116311539B true CN116311539B (en) | 2023-07-28 |
Family
ID=86801695
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310570275.8A Active CN116311539B (en) | 2023-05-19 | 2023-05-19 | Sleep motion capturing method, device, equipment and storage medium based on millimeter waves |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116311539B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116687357B (en) * | 2023-08-07 | 2023-10-20 | 亿慧云智能科技（深圳）股份有限公司 | Sleep monitoring method, device, equipment and storage medium based on millimeter wave radar |
| CN117524413B (en) * | 2024-01-05 | 2024-03-19 | 亿慧云智能科技（深圳）股份有限公司 | Motion protection method, device, equipment and medium based on millimeter wave radar |
Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102641125A (en) * | 2011-02-18 | 2012-08-22 | 西铁城控股株式会社 | Sleep breath pause judging device |
| CN102799856A (en) * | 2012-06-15 | 2012-11-28 | 天津大学 | Human action recognition method based on two-channel infrared information fusion |
| CN103761746A (en) * | 2013-12-25 | 2014-04-30 | 浙江慧谷信息技术有限公司 | Method for detecting periodical behavior based on similarity characteristics of areas of interest and reference frames selected according to extreme value |
| CN104463100A (en) * | 2014-11-07 | 2015-03-25 | 重庆邮电大学 | Intelligent wheelchair man-machine interaction system and method based on facial expression recognition mode |
| CN106264470A (en) * | 2016-09-12 | 2017-01-04 | 广东欧珀移动通信有限公司 | The method and device of sleep monitor |
| CN106897659A (en) * | 2015-12-18 | 2017-06-27 | 腾讯科技（深圳）有限公司 | The recognition methods of blink motion and device |
| CN107149465A (en) * | 2017-06-05 | 2017-09-12 | 深圳市爱贝宝移动互联科技有限公司 | A kind of sleep monitor system and its monitoring method |
| CN107735796A (en) * | 2016-10-31 | 2018-02-23 | 深圳市大疆创新科技有限公司 | Action identification method, network training method, device and equipment |
| CN108805088A (en) * | 2018-06-14 | 2018-11-13 | 南京云思创智信息科技有限公司 | Physiological signal analyzing subsystem based on multi-modal Emotion identification system |
| CN108804997A (en) * | 2018-03-29 | 2018-11-13 | 李金平 | Visual angle scene image processing platform |
| CN108899050A (en) * | 2018-06-14 | 2018-11-27 | 南京云思创智信息科技有限公司 | Speech signal analysis subsystem based on multi-modal Emotion identification system |
| CN108992040A (en) * | 2018-07-30 | 2018-12-14 | 深圳智芯数据服务有限公司 | A kind of sleep quality state monitoring method and device |
| CN110580741A (en) * | 2019-09-05 | 2019-12-17 | 深圳市赛为智能股份有限公司 | infant milk vomiting behavior detection method and device, computer equipment and storage medium |
| CN113408389A (en) * | 2021-06-10 | 2021-09-17 | 西华大学 | Method for intelligently recognizing drowsiness action of driver |
| CN113487522A (en) * | 2021-09-08 | 2021-10-08 | 深圳市诚识科技有限公司 | Multi-channel switching noise reduction method for image communication |
| CN114840684A (en) * | 2022-04-25 | 2022-08-02 | 平安普惠企业管理有限公司 | Map construction method, device and equipment based on medical entity and storage medium |
| CN115205225A (en) * | 2022-06-22 | 2022-10-18 | 平安科技（深圳）有限公司 | Training method, device and equipment of medical image recognition model and storage medium |
| CN115439850A (en) * | 2022-10-08 | 2022-12-06 | 招商局通商融资租赁有限公司 | Image-text character recognition method, device, equipment and storage medium based on examination sheet |
| CN115661531A (en) * | 2022-10-31 | 2023-01-31 | 龙瀛 | Image-text-based early oral cancer identification method, device, equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105493447B (en) * | 2013-07-01 | 2020-07-24 | 耐克创新有限合伙公司 | Wireless initialization of first-use electronic devices |
| US9585601B2 (en) * | 2013-08-15 | 2017-03-07 | Chunyuan Qiu | Systems, methods and devices for diagnosing sleep apnea |
-
2023
- 2023-05-19 CN CN202310570275.8A patent/CN116311539B/en active Active
Patent Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102641125A (en) * | 2011-02-18 | 2012-08-22 | 西铁城控股株式会社 | Sleep breath pause judging device |
| CN102799856A (en) * | 2012-06-15 | 2012-11-28 | 天津大学 | Human action recognition method based on two-channel infrared information fusion |
| CN103761746A (en) * | 2013-12-25 | 2014-04-30 | 浙江慧谷信息技术有限公司 | Method for detecting periodical behavior based on similarity characteristics of areas of interest and reference frames selected according to extreme value |
| CN104463100A (en) * | 2014-11-07 | 2015-03-25 | 重庆邮电大学 | Intelligent wheelchair man-machine interaction system and method based on facial expression recognition mode |
| CN106897659A (en) * | 2015-12-18 | 2017-06-27 | 腾讯科技（深圳）有限公司 | The recognition methods of blink motion and device |
| CN106264470A (en) * | 2016-09-12 | 2017-01-04 | 广东欧珀移动通信有限公司 | The method and device of sleep monitor |
| CN107735796A (en) * | 2016-10-31 | 2018-02-23 | 深圳市大疆创新科技有限公司 | Action identification method, network training method, device and equipment |
| CN107149465A (en) * | 2017-06-05 | 2017-09-12 | 深圳市爱贝宝移动互联科技有限公司 | A kind of sleep monitor system and its monitoring method |
| CN108804997A (en) * | 2018-03-29 | 2018-11-13 | 李金平 | Visual angle scene image processing platform |
| CN108805088A (en) * | 2018-06-14 | 2018-11-13 | 南京云思创智信息科技有限公司 | Physiological signal analyzing subsystem based on multi-modal Emotion identification system |
| CN108899050A (en) * | 2018-06-14 | 2018-11-27 | 南京云思创智信息科技有限公司 | Speech signal analysis subsystem based on multi-modal Emotion identification system |
| CN108992040A (en) * | 2018-07-30 | 2018-12-14 | 深圳智芯数据服务有限公司 | A kind of sleep quality state monitoring method and device |
| CN110580741A (en) * | 2019-09-05 | 2019-12-17 | 深圳市赛为智能股份有限公司 | infant milk vomiting behavior detection method and device, computer equipment and storage medium |
| CN113408389A (en) * | 2021-06-10 | 2021-09-17 | 西华大学 | Method for intelligently recognizing drowsiness action of driver |
| CN113487522A (en) * | 2021-09-08 | 2021-10-08 | 深圳市诚识科技有限公司 | Multi-channel switching noise reduction method for image communication |
| CN114840684A (en) * | 2022-04-25 | 2022-08-02 | 平安普惠企业管理有限公司 | Map construction method, device and equipment based on medical entity and storage medium |
| CN115205225A (en) * | 2022-06-22 | 2022-10-18 | 平安科技（深圳）有限公司 | Training method, device and equipment of medical image recognition model and storage medium |
| CN115439850A (en) * | 2022-10-08 | 2022-12-06 | 招商局通商融资租赁有限公司 | Image-text character recognition method, device, equipment and storage medium based on examination sheet |
| CN115661531A (en) * | 2022-10-31 | 2023-01-31 | 龙瀛 | Image-text-based early oral cancer identification method, device, equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 基于傅里叶-隐马尔科夫模型的人体行为识别方法研究;胡炜;;电子设计工程(07);第191-194页 * |
| 基于特征提取的步态识别技术;郭骁;;信息技术与信息化(08);第32-34页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116311539A (en) | 2023-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116311539B (en) | Sleep motion capturing method, device, equipment and storage medium based on millimeter waves | |
| CN113159147B (en) | Image recognition method and device based on neural network and electronic equipment | |
| CN110147732A (en) | Refer to vein identification method, device, computer equipment and storage medium | |
| CN114496264B (en) | Health index analysis method, device, equipment and medium based on multidimensional data | |
| CN113270197A (en) | Health prediction method, system and storage medium based on artificial intelligence | |
| WO2020224433A1 (en) | Target object attribute prediction method based on machine learning and related device | |
| CN113990482A (en) | Health data processing system and method | |
| CN116681923A (en) | Automatic ophthalmic disease classification method and system based on artificial intelligence | |
| CN117392470B (en) | Fundus image multi-label classification model generation method and system based on knowledge graph | |
| CN116687394B (en) | Tumble detection method, device, equipment and storage medium based on millimeter wave radar | |
| CN116942149B (en) | Lumbar vertebra monitoring method, device, equipment and storage medium based on millimeter wave radar | |
| CN116109977B (en) | Food sanitation detection method, device, equipment and medium based on production environment | |
| CN116257885A (en) | Private data communication method, system and computer equipment based on federal learning | |
| CN114973374A (en) | Expression-based risk evaluation method, device, equipment and storage medium | |
| CN114049676A (en) | Fatigue state detection method, device, equipment and storage medium | |
| CN117393109B (en) | Scene-adaptive diet monitoring method, device, equipment and storage medium | |
| CN113762042A (en) | Video identification method, device, equipment and storage medium | |
| CN113920590A (en) | Living body detection method, living body detection device, living body detection equipment and readable storage medium | |
| CN116687357B (en) | Sleep monitoring method, device, equipment and storage medium based on millimeter wave radar | |
| CN117253096B (en) | Finger-adaptive health index monitoring method, device, equipment and storage medium | |
| CN117392760B (en) | Health guidance method and system based on halved cross network | |
| CN111582404B (en) | Content classification method, device and readable storage medium | |
| CN113887408B (en) | Method, device, equipment and storage medium for detecting activated face video | |
| CN113537407B (en) | Image data evaluation processing method and device based on machine learning | |
| CN112819610B (en) | Credit evaluation method, credit evaluation model training method and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |