WO2024060076A1

WO2024060076A1 - Respiration monitoring method, apparatus and device, and computer-readable storage medium

Info

Publication number: WO2024060076A1
Application number: PCT/CN2022/120264
Authority: WO
Inventors: 鲍尚琦
Original assignee: 鲍尚琦; 张宝璂
Priority date: 2022-09-21
Filing date: 2022-09-21
Publication date: 2024-03-28

Abstract

The present application relates to the technical field of image processing, and in particular, to a respiration monitoring method, apparatus and device, and a computer-readable storage medium. The method comprises: acquiring, from a plurality of continuously acquired images, corresponding abdominal regions, respectively, each abdominal region comprising at least the abdomen of a target subject; determining a plurality of correlation values for each abdominal region; dividing the plurality of correlation values for each abdominal region to give a plurality of correlation value sets, each correlation value set representing one respiration completed by the target subject; and determining, according to the plurality of correlation value sets, whether the target subject respires normally or not. Therefore, the present application can acquire a plurality of correlation value sets according to the correlation between the abdominal regions, such that whether the target subject normally respires or not is determined according to the plurality of correlation value sets. The present application can improve the convenience of respiration monitoring, simplify the operation of users, and improve the accuracy of respiration monitoring, thus featuring suitability for various daily scenarios requiring the respiration monitoring of users.

Description

Respiratory monitoring methods, devices, equipment and computer-readable storage media

Technical field

The present application belongs to the field of image processing technology, and in particular relates to a respiratory monitoring method, device, equipment and computer-readable storage medium.

Background technique

With the continuous development of the economy, users have an increasingly strong need to know their health status at any time. Among them, respiratory rate, as an important parameter reflecting physiological conditions and sleep quality, can provide key information closely related to the user's health status. It can be seen that how to monitor respiratory frequency is very important.

At present, there are mainly two monitoring methods: contact and non-contact. In the contact monitoring method, the user needs to be in contact with the monitoring equipment or a special mattress to monitor the user's breathing. It is less comfortable and not suitable for users with fragile skin. In the non-contact monitoring method, the user's breathing is monitored through respiratory measurement methods such as bioradar method and thermal imaging method. Corresponding monitoring equipment and professionals are required, which is inconvenient for users' daily monitoring.

Therefore, how to monitor the user's breathing in daily scenarios is an urgent problem that needs to be solved.

Contents of the invention

This application provides a respiratory monitoring method, device, equipment and computer-readable storage medium, which can avoid the discomfort caused by the connection between the user and the monitoring equipment, and save professional human resources and medical equipment resources.

In the first aspect, this application provides a respiratory monitoring method, including:

From the continuously collected multi-frame images, obtain respective corresponding abdominal areas, and each abdominal area at least includes the abdomen of the target body;

Determine multiple correlation values for each abdominal region, and the multiple correlation values for any one abdominal region are used to represent the correlation between any one abdominal region and each of a preset number of abdominal regions, the preset number of The abdominal area includes any abdominal area and multiple abdominal areas collected continuously after any abdominal area. The preset number is greater than or equal to the total number of frames that can be collected after completing multiple breaths;

Divide multiple correlation values in each abdominal area to obtain multiple sets of correlation values, each set of correlation values is used to represent a breath completed by the target body;

Based on multiple sets of correlation values, determine whether the target body is breathing normally.

In a possible implementation manner of the first aspect, the method further includes:

Multiple respiratory frequencies are determined based on multiple sets of correlation values, and each respiratory frequency is used to indicate whether the target body is breathing normally or abnormally.

In a possible implementation of the first aspect, after determining multiple respiratory frequencies based on multiple sets of correlation values, the method further includes:

According to each respiratory frequency and the duration of the multi-frame image, a respiratory frequency curve is drawn. The respiratory frequency curve is used to indicate whether the target body is in an exhalation state or an inhalation state.

In a possible implementation of the first aspect, multiple respiratory frequencies are determined based on multiple sets of correlation values, including:

Determine the correlation value located at the preset position in each set of correlation values as the target correlation value;

determining a first abdominal region and a second abdominal region associated with the target correlation value in each set of correlation values;

Determine multiple target correlation values corresponding to each same first abdominal region;

For multiple target correlation values corresponding to any same first abdominal region, two second abdominal regions respectively associated with any two target correlation values are determined, and based on the difference between the two second abdominal regions The difference in frame numbers is used to obtain a respiratory rate.

In a possible implementation of the first aspect, when any same first abdominal region corresponds to multiple target correlation values, each group of correlation values to which the multiple target correlation values respectively belong is used to represent the normal breathing of the target body. ;

When any same first abdominal region corresponds to a target correlation value, a group of correlation values to which the target correlation value belongs is used to represent abnormal breathing of the target body.

In a possible implementation of the first aspect, the method further includes:

When it is determined that the target body's breathing is abnormal, a warning message indicating the target body's abnormal breathing is displayed.

In a possible implementation of the first aspect, corresponding abdominal regions are obtained from the continuously collected multiple frames of images, including:

Obtain multiple frames of images from the target’s breathing video;

Perform key point detection on each frame of image to obtain the key points of the neck, the key points of the left shoulder, the key points of the right shoulder and the key points of the mid-hip;

According to the key points of the neck, the left shoulder, the right shoulder and the middle hip, determine the key points of the right abdomen and the left abdomen;

Determine the abdominal area based on the key points of the left shoulder, the key points of the right shoulder, the key points of the right abdomen, and the key points of the left abdomen.

In a possible implementation of the first aspect, determining key points of the right abdomen and key points of the left abdomen according to key points of the neck, key points of the left shoulder, key points of the right shoulder, and key points of the mid-hip, includes:

On the connection line between the key point of the neck and the key point of the mid-hip, determine the position at a preset length from the key point of the mid-hip as the key point of the mid-abdomen;

Determine the key point of the right abdomen on the side where the key point of the mid-abdomen is close to the key point of the right shoulder. The distance between the key point of the mid-abdomen and the key point of the right abdomen is the same as the key point of the neck and the right shoulder. The distances between key points are equal;

On the side of the key point of the mid-abdomen close to the key point of the left shoulder, determine the key point of the left abdomen. The distance between the key point of the mid-abdomen and the key point of the left abdomen is the same as the key point of the neck and the key point of the left shoulder. The distance between points is equal.

In a possible implementation of the first aspect, determining the abdomen area according to the key points of the left shoulder, the key points of the right shoulder, the key points of the right abdomen, and the key points of the left abdomen includes:

Project the key points of the left shoulder, the key points of the right shoulder, the key points of the right abdomen and the key points of the left abdomen into the same coordinate system;

According to the coordinate system, determine the minimum abscissa, maximum abscissa, minimum ordinate, and maximum among the coordinates of the key point on the left shoulder, the coordinates of the key point on the right shoulder, the coordinates of the key point on the right abdomen, and the coordinates of the key point on the left abdomen. Y-axis;

Extend the minimum abscissa and the maximum abscissa along the vertical axis of the coordinate system respectively, and extend the minimum ordinate and the maximum ordinate along the horizontal axis of the coordinate system respectively to form a quadrilateral area;

The quadrilateral area was determined as the largest abdominal area.

The respiratory monitoring method provided by this application avoids the discomfort caused by the connection between the user and the monitoring equipment, saves the human resources and medical equipment resources of professionals, simplifies the user's operation, improves the convenience of respiratory monitoring, and ensures It improves the accuracy of respiratory monitoring results and is suitable for various daily scenarios that require monitoring the user's breathing.

In a second aspect, the present application provides a respiratory monitoring device, which is used to perform the respiratory monitoring method in the above first aspect or any possible implementation of the first aspect. Specifically, the device includes:

The acquisition module is used to obtain respective corresponding abdominal areas from the continuously collected multi-frame images, and each abdominal area at least includes the abdomen of the target body;

Determining module, used to determine multiple correlation values of each abdominal region, and multiple correlation values of any one abdominal region are used to represent the correlation between any one abdominal region and each abdominal region in a preset number of abdominal regions , the preset number of abdominal areas includes any abdominal area and multiple abdominal areas continuously collected after any one abdominal area, and the preset number is greater than or equal to the total number of frames of images that can be collected after completing multiple breaths;

The determination module is also used to divide multiple correlation values in each abdominal area to obtain multiple sets of correlation values, and each set of correlation values is used to represent a breath completed by the target body;

The determination module is also used to determine whether the target body is breathing normally based on multiple sets of correlation values.

In a third aspect, this application provides a device, which includes a memory and a processor. The memory is used to store instructions; the processor executes the instructions stored in the memory, so that the device performs the respiration monitoring method in the first aspect or any possible implementation of the first aspect.

In a fourth aspect, a computer-readable storage medium is provided. Instructions are stored in the computer-readable storage medium. When the instructions are run on a computer, they cause the computer to execute the first aspect or any possible implementation of the first aspect. Respiratory monitoring methods in.

A fifth aspect provides a computer program product containing instructions that, when run on a device, cause the device to execute the respiratory monitoring method in the first aspect or any possible implementation of the first aspect.

It can be understood that the beneficial effects of the above-mentioned second aspect to the fifth aspect can be referred to the relevant description in the above-mentioned first aspect, and will not be described again here.

Description of drawings

In order to explain the technical solutions in this application more clearly, the drawings needed to be used in the embodiments or description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some implementations of this application. For example, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a schematic flow chart of a respiratory monitoring method provided by an embodiment of the present application;

Figure 2 is a schematic diagram of correlation value calculation results provided by an embodiment of the present application;

FIG3 is a flow chart of a respiratory monitoring method provided in an embodiment of the present application;

Figure 4 is a schematic diagram of correlation value calculation results provided by an embodiment of the present application;

Figure 5 is a schematic diagram of a respiratory frequency curve provided by an embodiment of the present application;

FIG6 is a flow chart of a respiratory monitoring method provided in an embodiment of the present application;

Figure 7 is a schematic diagram of key points of the target provided by an embodiment of the present application;

Figure 8 is a schematic diagram of key points of the target provided by an embodiment of the present application;

Figure 9 is a schematic diagram of key points of the target provided by an embodiment of the present application;

FIG10 is a schematic diagram of the structure of a respiratory monitoring device provided in one embodiment of the present application;

Figure 11 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Reference signs:

Each solid point represents each key point of the target object in a frame of image;

Solid point 1 represents the key point of the neck;

Solid point 2 represents the key point of the right shoulder;

Solid point 3 represents the key point of the left button;

Solid point 4 represents the key point of the mid-hip;

The solid point 5 represents the key point of the midsection;

Solid point 6 represents the key point on the left abdomen;

Solid point 7 represents the key point on the right abdomen;

Rectangular frame A represents the abdominal area.

Detailed ways

In the following description, specific details, such as specific system structures and technologies, are provided for purposes of explanation and not limitation, in order to provide a thorough understanding of the present application. However, it will be apparent to those skilled in the art that the present application may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It will be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and/or components but does not exclude one or more other The presence or addition of features, integers, steps, operations, elements, components and/or collections thereof.

It will also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

In addition, in the description of this application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

Reference in this specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Therefore, the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in other embodiments", etc. appearing in different places in this specification are not necessarily References are made to the same embodiment, but rather to "one or more but not all embodiments" unless specifically stated otherwise. The terms “including,” “includes,” “having,” and variations thereof all mean “including but not limited to,” unless otherwise specifically emphasized.

This application proposes a respiratory monitoring method, device, equipment and computer-readable storage medium. The method is suitable for various scenarios requiring respiratory monitoring.

For example, the respiratory monitoring method of the present application is suitable for human body respiratory monitoring to facilitate determining whether the human body is breathing normally.

Based on the human body's sleep video, respiratory monitoring can be used to obtain the human body's breathing status, such as whether the human body is breathing normally and its breathing frequency. Furthermore, the health status of the human body can be estimated through the human body's breathing status, so as to facilitate the discovery of potential illnesses in the human body. , can be treated in time.

Among them, the respiratory monitoring method of the present application can be performed by electronic equipment.

Electronic devices can be smartphones, tablets, desktop computers, laptops, handheld devices, servers, etc.

In addition, the electronic device may include a display screen, or may be connected to an external display screen.

Therefore, the electronic device can display the respiratory monitoring results through the display screen, and the respiratory monitoring results can be whether the target body breathes normally, the breathing frequency of the target body, the breathing frequency curve of the target body, etc.

Based on the above scenario description, below, this application uses electronic equipment as an example to elaborate on the respiratory monitoring method provided by the embodiments of this application in conjunction with the drawings and application scenarios.

Please refer to FIG. 1 , which shows a schematic flow chart of a respiratory monitoring method provided by an embodiment of the present application.

As shown in Figure 1, the respiratory monitoring method provided by this application may include:

S101. Obtain corresponding abdominal areas from the continuously collected multiple frames of images.

The continuously acquired multiple frames of images refer to multiple frames of images that are continuous in time series, and each frame of image at least includes the abdominal area of the target body.

Therefore, the electronic device can acquire the abdominal area of each frame of the multi-frame images.

Each abdominal region includes at least the abdomen of the target body.

It should be understood that each time the subject completes a breath, the abdominal area rises and falls. Thus, the breathing process of the target body can be reflected through changes in the abdominal area.

S102. Determine multiple correlation values for each abdominal region.

Among them, the correlation value represents the degree of linear correlation between variables.

The multiple correlation values of any one abdominal region are used to represent the correlation between any one abdominal region and each of the preset number of abdominal regions.

The preset number of abdominal regions includes any abdominal region and multiple abdominal regions continuously acquired after any abdominal region, and the preset number is greater than or equal to the total number of frames of images that can be acquired by completing multiple breaths.

Assuming that the preset number of multi-frame images is 2550 frames, then there are 2550 abdominal areas, then the multiple correlation values of the first abdominal area include: the correlation value between the first abdominal area and the first abdominal area, The correlation value between the 1st abdominal area and the 2nd abdominal area... The correlation value between the 1st abdominal area and the 2550th abdominal area.

In some embodiments, the formula for calculating the correlation value between two abdominal regions is:

Among them, A _mn is a matrix of the abdominal region,

is the average of an abdominal region matrix, B _mn is the matrix of another abdominal region,

is the average of another abdominal region matrix.

S103. Divide multiple correlation values of each abdominal region to obtain multiple sets of correlation values.

Based on S102, multiple correlation values of each abdominal region can be obtained. Since multiple frames of images, that is, multiple abdominal regions, can be collected for each breath, multiple correlation values of multiple abdominal regions can form a set of correlations. value, so that the electronic device can obtain multiple sets of related values.

Each set of correlation values is used to characterize a breath completed by the target body.

As shown in Figure 2, the preset number is 2550 frames, and the horizontal and vertical coordinates are 2550 frames of continuously collected images, where each frame of image is associated with an abdominal region. Regarding the number of frames on the ordinate, it can be seen from Figure 2 that multiple correlation values of multiple abdominal areas can form a set of correlation values, that is, a gray area.

In Figure 2, when the ordinate is fixed, observe the increasing process of the abscissa. Each time it passes through a gray area, it can be considered that a breath has been completed.

For example, for 500 frames on the ordinate, the abscissa corresponding to 500 frames includes two gray areas, and the two gray areas respectively represent one breath completed by the target body.

S104. Determine whether the target body is breathing normally based on multiple sets of correlation values.

It should be understood that when the target body breathes normally, the state of the abdominal area can meet the preset breathing pattern. For example, every time the target body completes a breath, the abdominal area rises and falls once.

Then, according to the correlation between abdominal areas, each set of calculated correlation values must also satisfy similar correlation value rules. For example, every time the target body completes a breath, a set of correlation values can go through a process of first decreasing and then increasing. , or, every time the target body completes a breath, the number of frames of images that can be collected corresponding to a set of correlation values is within the preset range.

As a result, the electronic device can determine whether the target body is breathing normally based on the correlation value rules of each group of correlation values.

If this set of correlation values satisfies the correlation value rule, the electronic device can determine that the current target body is breathing normally.

If this set of correlation values does not satisfy the correlation value rules, the electronic device can determine that the current target body is breathing abnormally.

Continuing to combine with Figure 2, in terms of the number of frames on the ordinate, it can be seen from Figure 2 that each set of correlation values corresponding to the 1st frame to the 1500th frame satisfies the correlation value rule, and it can be determined that the target object is in the 1st frame The target body is breathing normally from the 1500th frame to the 1500th frame. The corresponding two sets of correlation values from the 1500th frame to the 2000th frame do not satisfy the correlation value rule. It can be determined that the target body is breathing abnormally at this time.

In addition, according to the calculation results and analysis of the S102 formula, it can be understood that when the abdominal area of the target body reaches the maximum bulge from the starting position, an inhalation process is completed. The abdominal area when the bulge is maximum is different from the starting position. The correlation value between the starting abdominal regions of the location is the smallest. When the abdominal area gradually drops back to the starting position from the maximum bulge, an exhalation process is completed. At this time, the position of the abdominal area is the same as the starting position of the starting abdominal area. Therefore, when the initial abdominal area is fixed, each time the correlation value first decreases and then increases, it can be considered that the target body has completed a breath.

In the respiratory monitoring method of this application, the electronic device obtains the corresponding abdominal areas from the continuously collected multiple frames of images, and can prepare the data for calculating multiple correlation values of each abdominal area through the correlation between the abdominal areas. . The electronic device can determine multiple correlation values for each abdominal area and divide the multiple correlation values for each abdominal area to obtain multiple sets of correlation values. The electronic device can determine whether each set of correlation values in the multiple sets of correlation values satisfies Correlation value rules determine whether the target body is currently breathing normally. Therefore, the electronic device can quickly and accurately determine whether the target body is breathing normally through continuously collected multi-frame images, simplifying the operation process, improving the efficiency of respiratory monitoring, ensuring the accuracy of respiratory monitoring results, and improving the convenience of respiratory monitoring. , suitable for various daily scenarios where user breathing needs to be monitored.

Based on the above description of the embodiment shown in Figure 1, the electronic device can also determine multiple breathing frequencies based on multiple sets of correlation values. Among them, each respiratory frequency can also be used to indicate whether the target body is breathing normally or abnormally.

Next, with reference to Figure 3, the specific implementation process of the respiratory monitoring method of the present application is introduced in detail.

Please refer to Figure 3. Figure 3 shows a schematic flow chart of a respiratory monitoring method provided by an embodiment of the present application.

As shown in Figure 3, the respiratory monitoring method provided by this application may include:

S201. Obtain corresponding abdominal areas from the continuously collected multiple frames of images.

S202. Determine multiple correlation values for each abdominal region.

S203. Divide multiple correlation values of each abdominal region to obtain multiple sets of correlation values.

Among them, S201, S202, and S203 are respectively implemented similarly to S101, S102, and S103 in the embodiment shown in FIG. 1, and will not be described again in this application.

S204. Determine the correlation value located at the preset position in each group of correlation values as the target correlation value.

Among them, the preset position is the center of gravity position of the gray area corresponding to each set of correlation values.

The target correlation value is the correlation value at the center of gravity of each group of correlation values.

In addition, the target correlation value can be understood as the minimum correlation value in the set of correlation values.

The electronic device determines the correlation value at the preset position as the target correlation value, and prepares the data for easy calculation of the respiratory frequency.

In addition, continuing with Figure 2, in order to facilitate the calculation of respiratory frequency, the electronic device can extract multiple sets of correlation values in Figure 2 through connected area technology to obtain Figure 4. As shown in Figure 4, the target correlation value is the correlation value of the center of gravity position of each black solid area.

S205. Determine the first abdominal region and the second abdominal region associated with the target correlation value in each set of correlation values.

It should be understood that each correlation value is calculated based on the correlation of two abdominal areas, therefore, each target correlation value may correspond to two abdominal areas, that is, a first abdominal area and a second abdominal area.

Continuing with Figure 4, it can be seen from Figure 4 that each target abdominal area corresponds to an abscissa and an ordinate, that is, there are two abdominal areas for each target correlation value.

S206. Determine multiple target correlation values corresponding to each same first abdominal area.

It should be understood that the preset number in S102 is greater than or equal to the total number of frames of images that can be collected after completing multiple breaths.

Thus, the number of multiple target correlation values is equal to the number of breaths.

For example, if the above multiple breaths are 2, then each same first abdominal area corresponds to 2 target related values.

Continuing to combine with Figure 4, it can be seen from Figure 4 that each same ordinate corresponds to 2 sets of correlation values, that is, corresponds to 2 target correlation values.

S207. For multiple target correlation values corresponding to any same first abdominal area, determine two second abdominal areas respectively associated with any two target correlation values, and determine the relationship between the two second abdominal areas according to The difference in the number of frames between them is used to obtain a respiratory frequency.

Based on S206, it can be determined that the number of multiple target related values is equal to the number of multiple breaths. Then, the difference in the number of frames between the two second abdominal regions is equivalent to the number of frames of images that can be collected by the target body after completing one breath.

In some embodiments, the calculation formula of respiratory frequency is: respiratory frequency = (60*f)/m.

Among them, m represents the difference in the abscissa between the centers of gravity of two adjacent groups of correlation values, that is, the frame number difference between two adjacent second abdominal regions, and f represents the frame rate.

Suppose m is 120 frames, that is, the number of frames that can be collected by the target body after completing one breath is 120 frames, and the frame rate is 30.

Then, respiratory frequency = (60*30)/120 = 15 times/min, that is, one breath every 4 seconds.

Therefore, the electronic device can calculate the breathing frequency of this breath based on the number of frames of images that can be collected when the target body completes one breath, so that the user can determine whether the target body is breathing normally based on the breathing frequency.

Wherein, when any same first abdominal region corresponds to multiple target correlation values, each group of correlation values to which the multiple target correlation values respectively belong is used to represent normal breathing of the target body.

When any same first abdominal area corresponds to a target correlation value, a group of correlation values to which one target correlation value belongs is used to represent abnormal breathing of the target body.

Continuing to combine with Figure 4, each same ordinate in Figure 4 corresponds to two target correlation values.

It can be seen from Figure 4 that when any same first abdominal area corresponds to two target correlation values, the two sets of correlation values to which the two target correlation values belong respectively are used to represent the normal breathing of the target body. In Figure 4, the target body breathes normally during the duration of the image from frame 1 to frame 1500.

When any same first abdominal area corresponds to one target correlation value, a group of correlation values to which one target correlation value belongs is used to indicate abnormal breathing of the target body. In Figure 4, if the target body corresponds to one target correlation value in multiple frames between the 1500th frame and the 2000th frame, then the electronic device can determine that the target body is breathing abnormally at this time.

In other words, the electronic device can determine whether the target body is breathing normally based on the number of target-related values corresponding to any same first abdominal area.

S208. When it is determined that the target body's breathing is abnormal, warning information indicating the target body's abnormal breathing may be displayed.

Among them, S208 is an optional step.

The electronic device may include a display interface, so that when the electronic device determines that the target body's breathing is abnormal, the electronic device may display warning information in the display interface. For example, display the alert "error". It is convenient to remind the user that the current target body is breathing abnormally.

This application does not limit the specific implementation of the display interface.

S209. Draw a respiratory frequency curve based on each respiratory frequency and the duration of the multi-frame image.

Among them, S209 is an optional step.

The respiratory frequency may be the respiratory frequency calculated in S207.

The duration of the multi-frame images may be the duration corresponding to the multiple frames of images continuously collected in S101.

The breathing rate curve is used to indicate whether the target is in an exhalation state or an inhalation state.

In some embodiments, the respiratory frequency curve is a sinusoidal function curve. Among them, the formula of the sine function is:

y=sin(wt);

w=π*k/30;

k*T=60;

T=2π/w.

Among them, t represents the duration corresponding to the continuously collected multiple frames of images, w represents the frequency of the sine function, T represents the period of the sine function, k represents the breathing frequency of k times/min, and each completed breath corresponds to one cycle of the sine function.

Figure 5 shows a sinusoidal function curve with a duration of 30 seconds corresponding to multiple frames of images collected continuously. The abscissa is time in seconds. The ordinate represents the breathing state of the target body. An increase represents inhalation and a decrease represents expiration. In Figure 5, the number of breaths of the target body in 20 seconds is 6, so the breathing rate is 18 times per minute, and the breathing frequency is 18 times/min. In addition, the electronic device can determine the respiratory state of the target body based on changes in the respiratory frequency curve in FIG. 5 .

Therefore, the electronic device can draw a respiratory frequency curve and display the respiratory frequency curve in the above-mentioned display interface. The electronic device uses the respiratory frequency curve to visualize the breathing status of the target body, allowing users to understand the breathing status of the target body more intuitively.

In this application, the electronic device can determine two second abdominal areas associated with any two target correlation values, and determine what the target body can do each time it completes one breath based on the frame number difference between the two second abdominal areas. Based on the frame number of the collected image, a respiratory frequency can be obtained. Therefore, the electronic device can use the respiratory frequency to determine whether the target body is breathing normally. In addition, the electronic device can display warning information when the target body is breathing abnormally, so as to promptly remind the user that the target body is breathing abnormally. In addition, the electronic device can also draw a respiratory frequency curve, allowing users to intuitively understand the respiratory status of the target body.

Based on the description of the embodiment shown in Figure 1 above, when the electronic device obtains the corresponding abdominal areas from the continuously collected multiple frames of images, it can use the key points of the neck of the target body, the key points of the left shoulder, and the key points of the right shoulder. Key points, and key points on the mid-hip, determine the abdominal area of the target body.

Next, with reference to Figure 6, the specific implementation process of the respiratory monitoring method of the present application will be introduced in detail.

The electronic device can obtain the key points of the target's neck, left shoulder, right shoulder, and mid-hip. According to the key points of the neck, left shoulder, right shoulder, and Determine the key points of the middle abdomen at the key points of the mid-hips, and then determine the key points of the right abdomen and the key points of the left abdomen on both sides of the key points of the mid-abdomen. According to the key points of the left shoulder, the key points of the right shoulder, and the key points of the right The key points of the abdomen, and the key points of the left abdomen, determine the abdominal area.

In some implementations, the electronic device can obtain the key points of each part of the target body through the human posture estimation algorithm, and determine the key points of the neck, the key points of the left shoulder, the key points of the right shoulder, and the key points of the target body from the key points of each part. The key point of the mid-buttocks.

As shown in Figure 7, each solid point is a key point of each part of the target body that the electronic device can obtain through the human posture estimation algorithm. Among them, solid point 1 is the key point of the neck, solid point 2 is the key point of the right shoulder, solid point 3 is the key point of the left shoulder, and solid point 4 is the key point of the middle hip.

The present application can store a human body posture estimation algorithm in an electronic device and/or a storage device that communicates with the electronic device, so that the electronic device can conveniently call the human body posture estimation algorithm to obtain key points of various parts of the target body.

Among them, this application does not limit the storage method and specific type of the storage device.

Please refer to FIG. 6 , which shows a schematic flowchart of a respiratory monitoring method provided by an embodiment of the present application.

As shown in Figure 6, the respiratory monitoring method provided by this application may include:

S301. Obtain multiple frames of images from the breathing video of the target body.

Among them, the breathing video of the target body is usually a video collected when the target body is in a sleep state.

It should be understood that when the target body is in a sleeping state, it rarely turns over or moves, and the abdominal area can be exposed, so that the electronic device can accurately monitor the breathing of the target body based on the ups and downs of the abdominal area.

In addition, the electronic device monitors the target's breathing based on the video collected when the target is sleeping, and can determine the target's sleep state and health information related to the sleep state.

The multi-frame image is a continuous multi-frame image of the breathing video of the target body in time sequence.

In some embodiments, the multi-frame images are multi-frame images that are continuous in time sequence in the entire breathing video of the target body.

In other embodiments, the multi-frame images are consecutive multi-frame images in time series captured from the entire breathing video of the target body.

In addition, the duration of the video collected when the above-mentioned target body is sleeping must be greater than 10 seconds. It can be understood that if the duration of the above video is less than 10 seconds, the number of breaths is too small, making it difficult to calculate accurate results.

S302. Perform key point detection on each frame of image to obtain the key points of the neck, the key points of the left shoulder, the key points of the right shoulder and the key points of the middle hip.

Among them, the electronic device uses a human posture estimation algorithm to detect key points in each frame of the image, and can obtain key points of multiple parts of the target body, and each key point is used to identify each part of the target body.

For example, the key points of multiple parts can include: key points of the head, key points of the neck, key points of the left shoulder, key points of the right shoulder, key points of the elbow, key points of the wrist, and key points of the middle hip. , the key points of the waist, the key points of the knees and other key points of various parts of the target body.

Furthermore, the electronic device can determine the key point of the neck, the key point of the left shoulder, the key point of the right shoulder, and the key point of the middle hip from the key points of multiple parts of the target body.

S303. On the connection line between the key point of the neck and the key point of the mid-hip, determine the position at a preset length from the key point of the mid-hip as the key point of the mid-abdomen.

Based on S302, the electronic device can obtain the key points of the neck, the key points of the left shoulder, the key points of the right shoulder, and the key points of the mid-hip. Therefore, the electronic device can determine the key point of the mid-abdomen on the connection line between the key point of the neck and the key point of the mid-hip.

In some embodiments, the key point of the mid-abdomen may be a point on the above-mentioned connection line that is a preset length away from the key point of the mid-buttocks.

Wherein, the preset length is predetermined. For example, the preset length can be 3/4 of the length from the key point of the neck to the key point of the mid-hip.

As shown in Figure 8, on the connection line between solid point 1 and solid point 4, the electronic device can determine the point at a preset length from the key point of the mid-hip, that is, solid point 5, as the key point of the mid-hip. .

S304. Determine the key point of the right abdomen on the side where the key point of the middle abdomen is close to the key point of the right shoulder.

S305. Determine the key point of the left abdomen on the side where the key point of the middle abdomen is close to the key point of the left shoulder.

Based on S303, the electronic device can obtain the key point of the mid-abdomen. Therefore, the electronic device can determine the key point of the right abdomen and the key point of the left abdomen on the left and right sides of the key point of the mid-abdomen.

Among them, the electronic device can calculate the distance between the key point of the neck and the key point of the right shoulder as the distance between the key point of the midsection and the key point of the right abdomen, and the distance between the key point of the neck and the key point of the left shoulder. The distance between, as the distance between the key point of the mid-abdomen and the key point of the left abdomen. That is, the distance between the key point of the midsection and the key point of the right abdomen is equal to the distance between the key point of the neck and the key point of the right shoulder. The distance between the keypoints on the midsection and the keypoints on the left abdomen is the same as the distance between the keypoints on the neck and the keypoints on the left shoulder.

Therefore, the electronic device can obtain accurate key points of the right abdomen and key points of the left abdomen.

In some embodiments, the electronic device can project the key points of multiple parts of the target body into the same coordinate system to obtain the key points of the neck, the key points of the left shoulder, the key points of the right shoulder, and the key points of the midsection. their respective coordinates. Therefore, the electronic device can calculate the coordinates of the key point on the right abdomen and the key point on the left abdomen.

Based on the above description, the electronic device can represent the key points of the neck, the key points of the left shoulder, the key points of the right shoulder, and the key points of the midsection through coordinates. The coordinates of the key points of the neck are (x ₁ , y ₁ ), The coordinates of the key point on the right shoulder are (x ₂ , y ₂ ), the coordinates of the key point on the left shoulder are (x ₃ , y ₃ ), and the coordinates of the key point on the midsection are (x ₄ , y ₄ ).

The above formula for calculating the coordinates of the key points on the right abdomen and the key points on the left abdomen is:

(x _a , y _a )=(x ₁ -x ₂ , y ₁ -y ₂ )

(x _b , y _b )=(x ₁ -x ₃ , y ₁ -y ₃ )

(x _c , y _c )=(x ₄ -x _a , y ₄ -y _a )

(x _d , y _d ) = (x ₄ −x _b , y ₄ −y _b )

Among them, (x _c , y _c ) is used to represent the coordinates of the key point on the right abdomen, and (x _d , y _d ) is used to represent the coordinates of the key point on the left abdomen.

Continuing to combine with Figure 8, the electronic device determines the key point of the right abdomen, that is, solid point 7, on the side of solid point 5 close to solid point 2, and determines the key point of the left abdomen on the side of solid point 5 close to solid point 3. , that is, solid point 6.

Thus, the electronic device can obtain the key points of the right abdomen and the key points of the left abdomen with the help of the key points of the middle abdomen, and prepare data for determining the abdominal area according to the key points of the right abdomen and the key points of the left abdomen.

S306. Project the key points of the left shoulder, the key points of the right shoulder, the key points of the right abdomen and the key points of the left abdomen into the same coordinate system.

S307. According to the coordinate system, determine the minimum abscissa, maximum abscissa, and minimum ordinate among the coordinates of the key point on the left shoulder, the coordinates of the key point on the right shoulder, the coordinates of the key point on the right abdomen, and the coordinates of the key point on the left abdomen. and the maximum ordinate.

Among them, the electronic device can project the key points of the left shoulder, the key points of the right shoulder, the key points of the right abdomen, and the key points of the left abdomen into the same coordinate system, and can obtain the coordinates of the key points of the left shoulder, the coordinates of the key points of the right shoulder, the coordinates of the key points of the right abdomen, and the coordinates of the key points of the left abdomen.

Therefore, the electronic device can determine the minimum abscissa, the maximum abscissa, and the minimum ordinate among the coordinates of the key point on the left shoulder, the coordinates of the key point on the right shoulder, the coordinates of the key point on the right abdomen, and the coordinates of the key point on the left abdomen. , and the maximum ordinate. The electronic device can prepare data for determining the abdominal area based on the minimum abscissa, the maximum abscissa, the minimum ordinate, and the maximum ordinate.

S308 , extending the minimum horizontal coordinate and the maximum horizontal coordinate along the vertical axis of the coordinate system respectively, and extending the minimum vertical coordinate and the maximum vertical coordinate along the horizontal axis of the coordinate system respectively, to form a quadrilateral area.

S309. Determine the quadrilateral area as the largest abdominal area.

As shown in FIG. 9 , the electronic device can determine the minimum abscissa, the maximum abscissa, the minimum ordinate, and the maximum ordinate in

solid points

2 , 3 , 6 , and 7 . It can be seen from Figure 9 that the minimum abscissa is the abscissa of solid point 6, the maximum abscissa is the abscissa of solid point 2, the minimum ordinate is the ordinate of solid point 7, and the maximum ordinate is the ordinate of solid point 3 coordinate.

Therefore, the electronic device can extend the minimum abscissa and the maximum abscissa along the vertical axis of the coordinate system, and the minimum ordinate and the maximum ordinate respectively along the horizontal axis of the coordinate system. During the extension process, the four lines can Intersect and connect to form a quadrilateral area.

Furthermore, the terminal device can determine the above-mentioned quadrilateral area as the largest abdominal area.

Continuing with FIG. 9 , it can be seen that the quadrilateral area formed in FIG. 9 is the largest abdominal area.

The electronic device obtains the largest abdominal area and can more accurately calculate the correlation value through the abdominal area. Therefore, the electronic device can determine the respiratory frequency of the target body based on the accurate correlation value data and determine whether the target body is breathing normally.

In this application, the electronic device can obtain the key points of the neck, the key points of the left shoulder, the key points of the right shoulder and the key points of the middle hip through key point detection, and between the key points of the neck and the key points of the middle hip Determine the key points of the mid-abdomen on the connecting line between them, and prepare the data for determining the key points of the right abdomen and the key points of the left abdomen on both sides of the key points of the mid-abdomen. The electronic device can determine the key point of the right abdomen on the side where the key point of the midsection is close to the key point of the right shoulder, and determine the key point of the left abdomen on the side where the key point of the midsection is close to the key point of the left shoulder. The device can quickly acquire the key points of the right abdomen and the key points of the left abdomen. Therefore, the electronic device can determine the abdominal area based on the key points of the left shoulder, the key points of the right shoulder, the key points of the right abdomen and the key points of the left abdomen. In addition, the electronic device can also determine the minimum abscissa, the maximum abscissa, and the minimum vertical coordinate based on the coordinates of the key point on the left shoulder, the coordinates of the key point on the right shoulder, the coordinates of the key point on the right abdomen, and the coordinates of the key point on the left abdomen. The coordinates and the maximum ordinate form a quadrilateral area, and this area is regarded as the largest abdominal area. Therefore, the electronic device can more accurately calculate the correlation value through the abdominal area based on the largest abdominal area.

Exemplarily, the present application also provides a respiratory monitoring device, which is applied to an electronic device.

Below, the respiratory monitoring device provided by an embodiment of the present application will be described in detail with reference to the figures.

Please refer to FIG. 10 , which shows a schematic block diagram of a respiratory monitoring device provided by an embodiment of the present application.

As shown in Figure 10, a respiratory monitoring device provided by an embodiment of the present application includes an acquisition module 401 and a determination module 402.

The acquisition module 401 is used to acquire respective corresponding abdominal areas from the continuously collected multi-frame images, and each abdominal area at least includes the abdomen of the target body;

The determination module 402 is used to determine multiple correlation values for each abdominal region. The multiple correlation values for any one abdominal region are used to represent the correlation between any one abdominal region and each abdominal region in a preset number of abdominal regions. The preset number of abdominal areas includes any abdominal area and multiple abdominal areas continuously collected after any one abdominal area. The preset number is greater than or equal to the total number of frames that can be collected after completing multiple breaths;

The determination module 402 is also used to divide multiple correlation values of each abdominal region to obtain multiple sets of correlation values, each set of correlation values used to represent a breath completed by the target body;

The determination module 402 is also used to determine whether the target body is breathing normally based on multiple sets of correlation values.

In some embodiments, the determining module 402 is specifically used to:

Continuing with reference to FIG. 10 , based on the structure shown in FIG. 10 , the respiratory monitoring device 400 may further include: a display module 403 . In Figure 10, the display module 403 is represented by a dotted line.

Among them, the display module 403 is used for:

A breathing frequency curve is drawn according to each breathing frequency and the duration of multiple frames of images. The breathing frequency curve is used to indicate whether the target is in an exhalation state or an inhalation state.

In some embodiments, the determination module 402 is specifically configured to:

In some embodiments, when any same first abdominal region corresponds to multiple target correlation values, each group of correlation values to which the multiple target correlation values respectively belong is used to represent normal breathing of the target body;

When any same first abdominal region corresponds to a target correlation value, a group of correlation values to which one target correlation value belongs is used to represent abnormal breathing of the target body.

In some embodiments, the display module 403 is specifically used for:

In some embodiments, the acquisition module 401 is specifically used to:

Obtain multiple frames of images from the breathing video of the target body;

Determine the key points of the right abdomen and the key points of the left abdomen based on the key points of the neck, the key points of the left shoulder, the key points of the right shoulder and the key points of the mid-hip;

In some embodiments, the acquisition module 401 is specifically used to:

The quadrilateral area was determined as the largest abdominal area.

It should be understood that the respiratory monitoring device 400 of the present application can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The above PLD can be a complex program logic device. (complex programmable logical device, CPLD), field-programmable gate array (FPGA), general array logic (generic array logic, GAL) or any combination thereof. The respiratory monitoring method in the above embodiment can also be implemented through software. When the respiratory monitoring method in the above embodiment is implemented through software, the respiratory monitoring device 400 and its respective modules can also be software modules.

By way of example, this application also provides a schematic structural diagram of an electronic device. As shown in Figure 11, the specific implementation of the electronic device can refer to the description of the above-mentioned electronic device, and can perform the above-mentioned respiratory monitoring method.

Among them, the electronic device 500 includes a processor 501, a memory 502, a communication interface 503 and a bus 504. Among them, the processor 501, the memory 502, and the communication interface 503 communicate through the bus 504. Communication can also be achieved through other means such as wireless transmission. The memory 502 is used to store instructions, and the processor 501 is used to execute the instructions stored in the memory 502 . The memory 502 stores program code 5021, and the processor 501 can call the program code 5021 stored in the memory 502 to execute the respiratory monitoring method in the above embodiment.

It should be understood that in the present application, the processor 501 may be a CPU, and the processor 501 may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

The memory 502 may include a read-only memory and a random access memory, and provide instructions and data to the processor 501. The memory 502 may also include a non-volatile random access memory. The memory 502 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM), and direct rambus RAM (direct rambus RAM, DR RAM).

In addition to a data bus, the bus 504 may also include a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, the various buses are labeled bus 504 in FIG. 11 .

It should be understood that the electronic device 500 of the present application may correspond to the electronic device in the above-described embodiment of the present application. When the electronic device 500 corresponds to the electronic device in the above-described embodiment, the above and other operations of each module in the electronic device 500 and The/or functions are respectively intended to implement the operating steps of the method executed by the electronic device in the above embodiments. For the sake of simplicity, they will not be described again here.

Exemplarily, this application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in each of the above method embodiments can be implemented.

Exemplarily, this application provides a computer program product. When the computer program product is run on an electronic device, the steps in each of the above method embodiments can be implemented when the electronic device is executed.

It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

It should be noted that the information interaction, execution process, etc. between the above-mentioned devices/units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional units and modules is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs. Module completion means dividing the internal structure of the above device into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be hardware-based. It can also be implemented in the form of software functional units. In addition, the specific names of each functional unit and module are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present application. For the specific working processes of the units and modules in the above system, please refer to the corresponding processes in the foregoing method embodiments, and will not be described again here.

In the above embodiments, each embodiment is described with its own emphasis. For parts that are not detailed or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed devices/network devices and methods can be implemented in other ways. For example, the device/network equipment embodiments described above are only illustrative. For example, the division of the above modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or units. Components may be combined or may be integrated into another system, or some features may be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

The units described above as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this application.

The above embodiments are only used to illustrate the technical solutions of the present application, but are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments. Modifications are made to the recorded technical solutions, or equivalent substitutions are made to some of the technical features; these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and shall be included in this application. within the scope of protection.

Claims

A respiratory monitoring method, comprising:

From the continuously collected multi-frame images, obtain respective corresponding abdominal areas, and each abdominal area at least includes the abdomen of the target body;

Determine multiple correlation values for each abdominal region, and the multiple correlation values for any one abdominal region are used to represent the correlation between any one abdominal region and each abdominal region in a preset number of abdominal regions, said The preset number of abdominal areas includes any one abdominal area and multiple abdominal areas continuously collected after any one abdominal area, and the preset number is greater than or equal to the total number of images that can be collected after completing multiple breaths. number of frames;

Divide multiple correlation values in each abdominal area to obtain multiple sets of correlation values, each set of correlation values used to characterize a breath completed by the target body;

According to the multiple sets of correlation values, it is determined whether the target body is breathing normally.
The method of claim 1, further comprising:

According to the plurality of sets of correlation values, a plurality of respiratory frequencies are determined, and each respiratory frequency is used to indicate whether the target body is breathing normally or abnormally.
The method according to claim 2, characterized in that, after determining a plurality of respiratory frequencies according to the plurality of sets of correlation values, further comprising:

According to each respiratory frequency and the duration of the multi-frame images, a respiratory frequency curve is drawn, and the respiratory frequency curve is used to indicate whether the target body is in an exhalation state or an inhalation state.
The method according to claim 2, characterized in that the step of determining a plurality of respiratory frequencies according to the plurality of sets of correlation values comprises:

Determine the correlation value at a preset position in each group of correlation values as a target correlation value;

determining a first abdominal region and a second abdominal region associated with the target correlation value in each set of correlation values;

Determine multiple target correlation values corresponding to each same first abdominal region;

For multiple target correlation values corresponding to any one of the same first abdominal regions, two second abdominal regions respectively associated with any two target correlation values are determined, and based on the two first abdominal regions, The frame number difference between the two abdominal regions is used to obtain a respiratory rate.
The method of claim 4, characterized in that:

When any one of the same first abdominal regions corresponds to a plurality of target correlation values, each group of correlation values to which the plurality of target correlation values belong respectively is used to represent normal breathing of the target body;

When any same first abdominal region corresponds to a target correlation value, a group of correlation values to which a target correlation value belongs is used to represent abnormal breathing of the target body.
The method according to any one of claims 1 to 5, characterized in that the method further includes:

When it is determined that the target body is breathing abnormally, warning information indicating that the target body is breathing abnormally is displayed.
The method according to any one of claims 1 to 5, characterized in that obtaining respective corresponding abdominal regions from the continuously collected multiple frames of images includes:

Obtain the multi-frame images from the breathing video of the target body;

Perform key point detection on each frame of image to obtain the key points of the neck, the key points of the left shoulder, the key points of the right shoulder and the key points of the mid-hip;

Determine the key points of the right abdomen and the key points of the left abdomen based on the key points of the neck, the key points of the left shoulder, the key points of the right shoulder and the key points of the middle hip;

The abdominal area is determined based on the key points of the left shoulder, the key points of the right shoulder, the key points of the right abdomen, and the key points of the left abdomen.
The method according to claim 7, wherein the right abdomen is determined based on the key points of the neck, the key points of the left shoulder, the key points of the right shoulder and the key points of the middle hip. key points and key points on the left abdomen, including:

On the connection line between the key point of the neck and the key point of the middle hip, determine a position that is a preset length away from the key point of the middle hip as the key point of the midsection;

Determine the key point of the right abdomen on the side of the key point of the mid-abdomen close to the key point of the right shoulder, where the distance between the key point of the mid-abdomen and the key point of the right abdomen is , equal to the distance between the key point of the neck and the key point of the right shoulder;

On the side of the key point of the mid-abdomen close to the key point of the left shoulder, determine the key point of the left abdomen, where the distance between the key point of the mid-abdomen and the key point of the left abdomen is, The distance between the key points of the neck and the key points of the left shoulder is equal.
The method according to claim 7, characterized in that, determining the key point of the left shoulder, the key point of the right shoulder, the key point of the right abdomen and the key point of the left abdomen. Abdominal area, including:

Project the key points of the left shoulder, the key points of the right shoulder, the key points of the right abdomen and the key points of the left abdomen into the same coordinate system;

According to the coordinate system, determine the minimum abscissa of the coordinates of the key point on the left shoulder, the coordinates of the key point on the right shoulder, the coordinates of the key point on the right abdomen, and the coordinates of the key point on the left abdomen. , the maximum abscissa, the minimum ordinate and the maximum ordinate;

The minimum abscissa and the maximum abscissa are respectively extended along the vertical axis of the coordinate system, and the minimum ordinate and the maximum ordinate are respectively extended along the horizontal axis of the coordinate system to form a quadrilateral. area;

The quadrilateral area is determined as the largest abdominal area.
A respiratory monitoring device, characterized by including:

The acquisition module is used to obtain respective corresponding abdominal areas from the continuously collected multi-frame images, and each abdominal area at least includes the abdomen of the target body;

Determining module, used to determine multiple correlation values of each abdominal region, and the multiple correlation values of any one abdominal region are used to represent the relationship between the any one abdominal region and each of the preset number of abdominal regions. Correlation, the preset number of abdominal areas includes any one abdominal area and multiple abdominal areas collected continuously after any one abdominal area, and the preset number is greater than or equal to what can be collected after completing multiple breaths The total number of frames of the image received;

The determination module is also used to divide multiple correlation values of each abdominal area to obtain multiple sets of correlation values, each set of correlation values used to characterize a breath completed by the target body;

The determination module is also used to determine whether the target body is breathing normally according to the multiple sets of correlation values.
An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the computer program, it implements claims 1 to 1 The method described in any one of 9.
A computer-readable storage medium, the computer-readable storage medium stores a computer program, characterized in that when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.