WO2023240510A1

WO2023240510A1 - Respiratory monitoring method and apparatus, earphone and storage medium

Info

Publication number: WO2023240510A1
Application number: PCT/CN2022/099022
Authority: WO
Inventors: 周岭松
Original assignee: 北京小米移动软件有限公司; 北京小米松果电子有限公司
Priority date: 2022-06-15
Filing date: 2022-06-15
Publication date: 2023-12-21
Also published as: CN117597941A

Abstract

Embodiments of the present disclosure provide a respiratory monitoring method and apparatus, an earphone and a storage medium. The respiratory monitoring method is applied to an earphone, and the earphone comprises a feedback microphone. The method comprises: collecting an audio signal in an ear canal by means of the feedback microphone to obtain an ear canal audio signal (S100); performing filtering processing on the ear canal audio signal to obtain a respiratory audio signal, wherein the respiratory audio signal is an audio signal generated when the vibration generated in the respiratory process of a user is transmitted to the ear canal by means of bone conduction when the earphone is worn by the user (S200); determining the respiratory rate of the user according to the respiratory audio signal (S300); and determining, according to the respiratory rate and a reference rate range, whether the respiratory rate is abnormal (S400).

Description

Respiratory monitoring method, device, earphone and storage medium

Technical field

The present disclosure relates to the field of information processing technology but is not limited to the field of information processing technology, and in particular, to a respiratory monitoring method, device, earphones and storage media.

Background technique

With the development of technology, more and more electronic devices have appeared in various application scenarios, and different electronic devices can achieve different functions in corresponding application scenarios.

With the widespread application of health monitoring equipment, the health status of the detected object can be detected through health monitoring equipment. Among them, breathing, as an important indicator of health status, plays an important role in determining health status. By monitoring the breathing, it can be determined whether the breathing of the monitored subject is normal.

Contents of the invention

Embodiments of the present disclosure provide a respiratory monitoring device, an earphone, and a storage medium.

A first aspect of an embodiment of the present disclosure provides a respiratory monitoring method, which is applied to an earphone, the earphone includes a feedback microphone, and the method includes:

Collect the audio signal in the ear canal through the feedback microphone to obtain the ear canal audio signal;

The ear canal audio signal is filtered to obtain a respiratory audio signal; wherein the respiratory audio signal is the vibration generated by the user during the breathing process when the earphone is worn by the user through bone conduction. The audio signal produced by being delivered to the ear canal;

Determine the user's respiratory frequency based on the respiratory audio signal;

According to the respiratory frequency and the reference frequency range, it is determined whether the user's respiratory frequency is abnormal.

A second aspect of the embodiment of the present disclosure provides a respiratory monitoring device, applied to earphones, where the earphones include a feedback microphone; the device includes:

An ear canal audio signal collection module is configured to collect the audio signal in the ear canal through the feedback microphone to obtain the ear canal audio signal;

The respiratory audio signal determination module is configured to perform filtering processing on the ear canal audio signal to obtain a respiratory audio signal; wherein the respiratory audio signal is when the earphone is worn by the user and the user is breathing. The vibrations generated during the process are transmitted to the ear canal through bone conduction to generate audio signals;

a respiratory frequency determination module configured to determine the respiratory frequency of the user based on the respiratory audio signal;

An abnormality determination module is configured to determine whether the respiratory frequency is abnormal according to the respiratory frequency and a reference frequency range.

A third aspect of the present disclosure provides an earphone. The earphone includes a housing and a controller, a feedback microphone, a feedforward microphone and a speaker provided on the housing; the feedforward microphone is connected to the controller. , used to collect audio data outside the ear canal and send it to the controller; the feedback microphone is connected to the controller, used to collect audio data inside the ear canal and send it to the controller; the controller includes a memory and a processor The processor has executable computer instructions stored on the memory, and the processor can call the computer instructions stored on the memory to execute the respiratory monitoring method as provided in the first aspect when executing the program.

A fourth aspect of the embodiments of the present disclosure provides a computer storage medium that stores an executable program; after the executable program is executed by a processor, the respiratory monitoring method provided by the first aspect can be implemented.

The respiratory monitoring method provided by the embodiments of the present disclosure can be applied to headphones, and the user's respiratory frequency can be determined through the headphones without the need for other monitoring sensors, thereby improving the convenience of monitoring the user's respiratory frequency and improving the user's experience.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the embodiments of the present disclosure.

Description of the drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description serve to explain the principles of the embodiments of the invention.

Figure 1 is a schematic diagram of a respiratory monitoring method according to an exemplary embodiment;

Figure 2 is a schematic diagram of an earphone according to an exemplary embodiment;

Figure 3 is a schematic diagram of an earphone in a state of being worn by a user according to an exemplary embodiment;

Figure 4 is a schematic diagram of another method for determining respiratory monitoring according to an exemplary embodiment;

Figure 5 is a schematic diagram illustrating a method of determining respiratory frequency according to an exemplary embodiment;

Figure 6 is a schematic diagram of determining a target amplitude according to an exemplary embodiment;

Figure 7 is a schematic diagram illustrating a method of determining a target frequency range and a reference frequency range according to an exemplary embodiment;

Figure 8 is a schematic diagram illustrating a method of determining a target frequency range according to an exemplary embodiment;

Figure 9 is a schematic diagram illustrating a method of determining a reference frequency range according to an exemplary embodiment;

Figure 10 is a schematic structural diagram of a respiratory monitoring device according to an exemplary embodiment;

FIG. 11 is a schematic structural diagram of an electronic device according to an exemplary embodiment.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with embodiments of the invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of embodiments of the invention.

The terminology used in the embodiments of the present disclosure is for the purpose of describing specific embodiments only and is not intended to limit the embodiments of the present disclosure. As used in this disclosure, the singular forms "a," "the" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present disclosure, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the embodiments of the present disclosure, the first information may also be called second information, and similarly, the second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "when" or "in response to determining."

Under normal circumstances, respiratory frequency can reflect physical health. When monitoring respiratory frequency, professional monitoring equipment is required. For example, when a user goes to the hospital for medical treatment, the doctor can monitor the user's respiratory frequency through equipment used to monitor breathing in the hospital, such as piezoelectric sensors, thermal resistance sensors and/or infrared sensors for respiratory monitoring. These sensors are deployed in corresponding parts. When the human body breathes, some fluctuations occur in the parts of the human body where the sensors are deployed. These sensors can generate electrical signals based on these fluctuations, and thus the respiratory frequency can be determined based on these electrical signals. Different breathing rates can be identified by analyzing the signals sensed by different sensors. And the relevant operations in this process need to be completed by professional medical staff.

When monitoring the user's respiratory frequency through these sensors, the user needs to wear multiple sensors in a specific place and in a specific state to monitor the respiratory frequency. For example, in the corresponding department of the hospital, the user completes it in a non-moving state, such as a stationary state. In this way, the respiratory frequency cannot be monitored anytime and anywhere, which reduces the convenience of monitoring the respiratory frequency, thereby reducing the user experience.

Referring to FIG. 1 , a schematic diagram of a respiratory monitoring method provided by an embodiment of the present disclosure is shown. The method can be applied to at least earphones, and the earphones can at least include a feedback microphone.

As shown in Figure 1, the method includes:

Step S100, collect the audio signal in the ear canal through the feedback microphone to obtain the respiratory audio signal;

Step S200, perform filtering processing on the ear canal audio signal to obtain a respiratory audio signal; wherein, the respiratory audio signal is when the earphone is worn by the user, and the vibration generated by the user during breathing is transmitted to the ear canal through bone conduction. The audio signal produced.

Step S300: Determine the user's breathing frequency based on the breathing audio signal.

Step S400: Determine whether the respiratory frequency is abnormal based on the respiratory frequency and the reference frequency range.

Earphones can be of different shapes, including in-ear, semi-in-ear, and head-mounted earphones. The communication method of the headset may include wired headsets and wireless headsets, and the wireless headsets may include Bluetooth headsets, such as True Wireless Stereo headsets (TWS). Headphones may also include devices such as hearing aids that have feedback microphones and are capable of implementing this scheme.

The feedback microphone in the headset can be located near the sound channel of the headset. When the headset is worn, the feedback microphone is located in the ear canal and can collect audio signals in the ear canal, such as in-ear headphones. When the earphones are other types of earphones, such as semi-in-ear earphones and headphones, the feedback microphone can collect audio signals in the ear canal when the earphones are worn.

Referring to Figure 2, Figure 2 is a schematic diagram of an earphone, including a feedback microphone a. When the earphone is worn, the feedback microphone a can be located in the ear canal. The headset may also include a feedforward microphone b, which may be located on the stem of the headset. When the headset is worn, the feedforward microphone is located outside the ear canal and can collect environmental audio signals from the external environment. The headset may also include a call microphone c to collect audio signals sent by the user during a call. The signal-to-noise ratio of the audio signal collected by feedback microphone a is higher than that of feedforward microphone b, so an ear canal audio signal with less noise and higher quality is collected through feedback microphone a.

When the earphone is a headset, a certain degree of ear-blocking effect will also occur. When the earphone is worn, the feedback microphone a can be located outside the ear canal or toward the ear canal, and can collect ear canal audio signals.

Referring to Figure 3, Figure 3 is a schematic diagram of an earphone when being worn by a user. The earphone 1 blocks the ear canal 2 to a certain extent, forming an ear-blocking effect. The ear canal audio signals collected by the feedback microphone include: when the headset is worn by the user, the vibration generated by the user during breathing is transmitted to the ear canal through bone conduction, that is, the audio signal 3.

For step S100, when the earphones are in the wearing state, the ear canal will be blocked to a certain extent, resulting in a certain degree of ear blocking effect. The reason for this is that some sounds will be transmitted to the inner ear through the human bones. For example, because the nasal cavity is far away from the ear canal, Recently, when the user breathes, vibrations are generated between the airflow and the respiratory tract, and the vibrations are transmitted to the audio signals in the ear canal through bone conduction. When the earphones are not worn, part of the sound from the bone conduction diffuses outward through the outer ear. However, when the earphones are worn, the ear canal is blocked to a certain extent, which reduces the sound from the bone conduction to the outside through the ear canal. The amount of diffusion creates a certain degree of ear-blocking effect, also known as the occlusion effect. The sound characteristics produced by the occlusion effect are characterized by strengthening low-frequency signals and weakening high-frequency signals.

Because the earphones block the ear canal to a certain extent, they form varying degrees of ear blocking effect. After the ear blocking effect occurs, the earphones block external audio signals from entering the ear canal, reducing the impact of external audio signals on the audio signals in the ear canal. The feedback microphone can collect the audio signal in the ear canal to obtain the ear canal audio signal.

For step S200, the ear canal audio signal is filtered to obtain a respiratory audio signal. Since the ear canal audio signal includes not only the respiratory audio signal, but also other audio signals, the respiratory audio signal can be obtained by filtering the ear canal audio signal. Filtering can be done based on frequency through filtering algorithms, or filtering can be done through filters.

The respiratory audio signal is the ear canal audio signal, which includes the audio signal generated when the earphone is worn by the user and the vibration generated by the user's breathing during the breathing process is transmitted to the ear canal through bone conduction. The vibrations caused by the user's breathing during the breathing process may include vibrations of the airflow in the respiratory tract and/or vibrations of the respiratory tract, which may include the nasal cavity.

After a certain degree of ear-blocking effect is formed, the vibrations generated by the user during breathing can be transmitted to the ear canal through bone conduction, generating respiratory audio signals, such as audio signals in the range of 0.1Hz to 10Hz. The ear-blocking effect can amplify this frequency range. The audio signal makes it easy for the feedback microphone to collect the respiratory audio signal. This range can be adjusted according to the user's motion status.

For step S300, after obtaining the respiratory audio signal, the user's respiratory frequency can be determined based on the respiratory audio signal. The breathing audio signal can reflect the user's breathing status. The breathing audio signal can be an audio signal in the time domain. The breathing audio signal can be time-frequency converted, and the conversion result can be analyzed to obtain the user's breathing frequency. The method of obtaining the user's breathing frequency based on the breathing audio signal is within the protection scope of this embodiment. For the detailed determination process, please refer to subsequent embodiments.

For step S400, after determining the respiratory frequency, it may be determined whether the respiratory frequency is abnormal based on the respiratory frequency and the reference frequency range. When the respiratory frequency exceeds the reference frequency range, the respiratory frequency is determined to be abnormal, and when the respiratory frequency is within the reference frequency range, the respiratory frequency is determined to be normal.

The reference frequency range can be determined based on the actual usage scenario and the state of the user's body. The state of the user's body is the physical state, which can include a moving state and a non-moving state. The non-exercise state may include a state where the user is sitting quietly, standing, or sleeping. When the user is sitting quietly or standing, the first reference frequency range corresponds to it, and when the user is sleeping, the second reference frequency range corresponds to it. The motion state may include states of different sports such as running, playing ball, swimming, etc. The running state corresponds to the third reference frequency range, and the playing state corresponds to the fourth reference frequency range.

According to the respiratory frequency matching the corresponding reference frequency range, when the respiratory frequency is within the corresponding reference frequency range, the respiratory frequency is normal; when the respiratory frequency is outside the corresponding reference frequency range, the respiratory frequency is abnormal.

This disclosed example can obtain the user's breathing audio signal through the earphones, and then detect the user's breathing frequency based on the breathing audio signal, and then determine whether the user's breathing frequency matches the corresponding reference frequency range, thereby determining whether the user's breathing frequency is abnormal. . Utilizing the existing built-in microphone, the respiratory rate can be determined through the earphones without the cost of new hardware or the use of various other sensors and other monitoring equipment. It reduces the difficulty and inconvenience of monitoring the user's breathing frequency, improves the convenience of monitoring the user's breathing frequency, and improves the user's experience.

In one embodiment, refer to FIG. 4 , which is a schematic diagram of a method for obtaining a respiratory audio signal. The method includes:

Step S201: The ear canal audio signal is divided into frames to obtain the ear canal audio signal divided into multiple frames.

Step S202: Filter the ear canal audio signal obtained after multiple frames are divided into multiple frames to obtain a multi-frame respiratory audio signal.

After collecting the audio signal in the ear canal through the feedback microphone in the earphone, the ear canal audio signal can be obtained. In order to facilitate subsequent processing, the audio signal in the ear canal collected by the feedback microphone is divided into frames to obtain the multi-frame divided audio signal. Ear canal audio signal. The multi-frame ear canal audio signal obtained after framing is more stable and continuous, which facilitates subsequent processing.

The frame length of the divided ear canal audio signal can be determined according to actual needs, such as 16ms, 32ms or 64ms. In order to make the transition between each frame of audio signal smooth and maintain its continuity, when dividing frames, there can be a frame shift between two adjacent frames, and the frame shift can be half the frame length. The frame length of each frame of the audio signal after framing may be the same.

Since other sounds can also be transmitted into the ear canal through bone conduction, the feedback microphone can also collect audio signals other than breathing audio signals, such as audio signals from running, walking, talking, tooth collisions, and head collisions with other things. . The ear canal audio signals collected by the feedback microphone may include respiratory audio signals, and may also include other audio signals besides the respiratory audio signals. The other audio signals except the respiratory audio signals are noise signals corresponding to the respiratory audio signals. Before determining the respiratory frequency based on the respiratory audio signal, these noise signals need to be filtered out.

Since the frequency of the respiratory audio signal is different from the frequency of other audio signals, the ear canal audio signal can be filtered to filter out other audio signals except the respiratory audio signal. For example, the frequency range of the respiratory audio signal can be in the range of 0.1Hz to 5Hz, with a concentration around 0.5Hz. The frequencies of other audio signals such as running, walking, talking, tooth collisions, and head collisions with other things are not within this range, or have less intersection with this frequency range. The frequencies of audio signals such as running, walking, or talking are usually greater than breathing audio. The frequency of the signal, and the amplitude of audio signals such as running, walking, or talking, is also greater than the amplitude of the breathing audio signal. Noise signals can be filtered out based on frequency. The frequency range of the breathing audio signal can be adjusted according to the user's physical condition.

The filtered ear canal audio signal is the respiratory audio signal, which can reduce the interference of the noise signal on the respiratory audio signal, thereby reducing the interference on the determined respiratory frequency and improving the accuracy of the determined respiratory frequency.

The filtering methods may include multiple methods. For example, the audio signal after each frame is divided into frames may be filtered through Fourier transform to obtain a multi-frame breathing audio signal. The audio signal after each frame is divided into frames can also be filtered through wavelet transform to obtain a multi-frame respiratory audio signal. The method of filtering the audio signals divided into frames and filtering audio signals other than the respiratory audio signals to obtain multi-frame respiratory audio signals is within the scope of this embodiment.

In one embodiment, refer to FIG. 5 , which is a schematic diagram for determining respiratory frequency. Step S300, determine the user's breathing frequency based on the breathing audio signal, including:

Step S301, determine the spectrum information of the respiratory audio signal;

Step S302: Based on the spectrum information, detect the target amplitude in the respiratory audio signal within the target frequency range; where the target amplitude is the maximum peak value of the fundamental wave of the respiratory audio signal;

Step S303: Determine the respiratory frequency according to the frequency corresponding to the target amplitude.

After the respiratory audio signal is determined, the respiratory audio signal is processed to determine the spectrum information of the respiratory audio signal. Since the respiratory audio signal collected through the feedback microphone is an audio signal in the time domain, the respiratory audio signal in the time domain can be converted to obtain an audio signal in the frequency domain, thereby obtaining the corresponding spectrum information. For example, by converting the respiratory audio signal in the time domain to the respiratory audio signal in the frequency domain through Fourier transform, the spectrum information of the respiratory audio signal can be obtained.

The spectrum information includes corresponding information of frequency and amplitude. According to the spectrum information, the target amplitude in the respiratory audio signal corresponding to the spectrum information can be detected within the target frequency range. The target amplitude is within the target frequency range, and the spectrum information corresponds to the maximum peak-to-peak value of the fundamental wave of the respiratory audio signal, that is, the maximum amplitude of the fundamental wave corresponds to the amplitude value. The fundamental wave of the respiratory audio signal can best reflect the spectrum information of the respiratory audio signal. Determining the respiratory frequency based on the frequency corresponding to the maximum peak value of the fundamental wave of the respiratory audio signal can improve the accuracy of the respiratory frequency. The target frequency range may be preset, or the corresponding target frequency range may be determined according to the user's physical condition.

In the spectrum information, the greater the amplitude, the greater the intensity of the corresponding audio signal. At the target amplitude, the intensity of the respiratory audio signal is maximum, so according to the frequency corresponding to the target amplitude in the spectrum information, the accuracy of determining the respiratory frequency is higher. The larger the amplitude, the higher the peak value of the wave peak. The maximum peak value of the fundamental wave within the target frequency range corresponds to the amplitude of the maximum amplitude of the fundamental wave. Determining the respiratory frequency based on the frequency corresponding to the target amplitude can improve the accuracy of determining the respiratory frequency.

In another embodiment, step S301 can also process each frame of respiratory audio signal in frame units to determine the spectrum information of each frame of respiratory audio signal, so that each frame of respiratory audio signal can be obtained. The spectrum information of the signal is used to determine the respiratory frequency based on the spectrum information of at least one frame of respiratory audio signal.

Step S302: The target amplitude corresponding to each frame of the respiratory audio signal can also be determined within the target frequency range based on the spectrum information of the respiratory audio signal of each frame.

Step S303, you can also determine the frequency corresponding to the target amplitude of each frame of respiratory audio signal based on the spectrum information corresponding to each frame of respiratory audio signal, and determine the respiratory frequency based on the frequency corresponding to the target amplitude of each frame of respiratory audio signal. .

In another embodiment, refer to FIG. 6 , which is a schematic diagram of determining a target amplitude. Step S302, based on the spectrum information, detect the target amplitude in the respiratory audio signal within the target frequency range, including:

S3021, based on spectrum information, detect the maximum peak value within the target frequency range.

S3022: Determine whether there is a waveform of the second waveform frequency within the target frequency range according to the first waveform frequency of the waveform corresponding to the maximum peak value; wherein the first waveform frequency is an integer multiple of the second waveform frequency.

S3023: When it is determined that there is a waveform of the second waveform frequency, determine the waveform of the second waveform frequency as the fundamental wave.

S3024, when it is determined that there is no waveform of the second waveform frequency, determine the waveform of the first waveform frequency as the fundamental wave;

S3025: Determine the maximum peak value of the fundamental wave as the target amplitude.

After obtaining the spectrum information of the respiratory audio signal, the waveform of the respiratory audio signal can be determined based on the spectrum information. Since the waveform has a peak, the maximum peak value is detected within the target frequency range according to the waveform in the spectrum information. In the spectrum information, the maximum peak value is The intensity of the respiratory audio signal is the largest at the peak, which best reflects the respiratory audio signal. According to the spectrum information, within the target frequency range, the frequency of the waveform corresponding to the maximum peak value can also be determined, and this frequency is recorded as the first waveform frequency. Then detect whether there is a waveform of the second waveform frequency within the target frequency range. The first waveform frequency is an integer multiple of the second waveform frequency. If a waveform of the second waveform frequency is detected, it is determined that there is a waveform of the second waveform frequency. Description The waveform of the first waveform frequency is not the fundamental wave of the respiratory audio signal. For example, the waveform of the first waveform frequency is the harmonic of the fundamental wave of the respiratory audio signal. The frequency of the harmonic is an integer multiple of the fundamental frequency, and the frequency of the harmonic is greater than the fundamental wave. The frequency and harmonics may have an adverse effect on determining the target amplitude, thereby affecting the determination of the respiratory frequency, then the waveform of the second waveform frequency is determined as the fundamental wave of the respiratory audio signal.

If the waveform of the second waveform frequency is not detected, it is determined that there is no waveform of the second waveform frequency, indicating that the waveform of the first waveform frequency is the fundamental wave of the respiratory audio signal within the target frequency range, then the waveform of the first waveform frequency is determined is the fundamental wave.

For example, in the spectrum information of the respiratory audio signal, there are two identical target amplitudes, the peaks of the two wave peaks are the same, and the frequencies corresponding to each target amplitude are different. The frequency corresponding to one target amplitude is 1Hz, and the frequency corresponding to the other target amplitude is 1Hz. The frequency corresponding to the value is 1.5Hz, then the frequency 1Hz is determined as the respiratory frequency.

In another embodiment, step S303, determining the respiratory frequency according to the frequency corresponding to the target amplitude, includes:

When the number of frames of the respiratory audio signal is multiple frames, the average frequency corresponding to the target amplitude in the multi-frame respiratory audio signal is determined as the respiratory frequency.

In the multi-frame respiratory audio signal, the target amplitude in each frame of the respiratory audio signal is determined based on the spectrum information of each frame of the respiratory audio signal, and then the average value of the frequencies corresponding to the target amplitude in the multi-frame respiratory audio signal is determined as Respiratory rate. This can reduce the impact on the accuracy of determining the respiratory frequency when individual target amplitude differences are large, thereby improving the accuracy of determining the respiratory frequency.

For example, if the number of frames of the respiratory audio signal is 10 frames, then the target amplitude of each frame of the respiratory audio signal is determined based on the spectrum information of each frame of the respiratory audio signal in the 10 frames of respiratory audio signal. According to the spectrum information of the respiratory audio signal of the first frame, the target amplitude 1 of the respiratory audio signal of the first frame is determined. According to the spectrum information of the respiratory audio signal of the second frame, the target amplitude 2 of the respiratory audio signal of the second frame is determined... By analogy, based on the spectrum information of the respiratory audio signal of the tenth frame, the target amplitude 10 of the respiratory audio signal of the tenth frame is determined. Then, within the target frequency range, the frequencies corresponding to the target amplitude 1 to the target amplitude 10 are determined, and the average frequency of these ten frequencies is determined as the respiratory frequency.

In another embodiment, refer to FIG. 7 , which is a schematic diagram for determining a target frequency range and a reference frequency range. The method includes:

Step A, obtain the user's physical state; the physical state includes a motion state and a non-motion state, the motion state includes at least one state corresponding to motion, and the non-motion state includes at least one state corresponding to non-motion;

Step B: Determine the target frequency range and reference frequency range according to the physical state. Different body states correspond to respective target frequency ranges, and different body states correspond to respective reference frequency ranges; in the same body state, the reference frequency range is included in the target frequency range.

The body status can be obtained through an external device connected to the headset. The external device sends the body status information to the headset after determining the body status, or it can be determined through the headset. The process of determining the body state is not limited; here, only the result of the acquired body state is used.

The target frequency range and the reference frequency range can be determined according to the user's physical state. When the user's physical state is different, the corresponding target frequency range and reference frequency range will also be different. The user's physical state can include a moving state and a non-moving state. The moving state can include states corresponding to various sports such as walking, running, playing ball, and swimming. The non-moving state can include non-moving states such as sitting still, standing, and sleeping.

According to the body state, the target frequency range and the reference frequency range that match the user's current body state can be determined. The target frequency range and reference frequency range corresponding to different physical states can be determined according to actual use requirements. When the age of the user is different, the target frequency range and the reference frequency range may be different under the same physical condition. The target frequency range and reference frequency range will also be different in different motion states. For example, the target frequency range and reference frequency range corresponding to the walking state are different from the target frequency range and reference frequency range corresponding to the running state. The target frequency range and reference frequency range may also be different in different non-exercise states. For example, the target frequency range and reference frequency range corresponding to the sleeping state may be different from the target frequency range and reference frequency range corresponding to the sitting state.

In the same body state, the reference frequency range is included in the target frequency range, the maximum value of the reference frequency range is less than the maximum value of the target frequency range, and the minimum value of the reference frequency range is less than the minimum value of the target frequency range. If this can reduce the respiratory frequency determined within the target frequency range to always be within the reference frequency range, the respiratory frequency will not exceed the reference frequency range, thereby reducing the situation where the respiratory frequency cannot be determined to be abnormal. It can also reduce the partial intersection between the reference frequency range and the target frequency range, resulting in a higher probability that the respiratory frequency will appear in an interval that does not intersect with the target frequency in the reference frequency range, resulting in a lower accuracy in determining whether the respiratory frequency is abnormal. problem to improve the accuracy of determining whether respiratory rate is abnormal.

In another embodiment, refer to FIG. 8 , which is a schematic diagram of determining a target frequency range. In step B, the target frequency range is determined based on the user's physical condition, including:

Step B1: Determine the first compensation value of the maximum value and the second compensation value of the minimum value in the target frequency range corresponding to different body states.

Step B2: Determine the target frequency range based on the first reference frequency range, the first compensation value and the second compensation value.

The target frequency range has a first reference frequency range. The first reference frequency range is used to determine the target frequency range under different body states. Different body states correspond to a first compensation value of the maximum value and a minimum value of the target frequency range. 2. Compensation value. The first compensation value and the second compensation value corresponding to different body states are different. The target frequency range matching the body state can be determined based on the first reference frequency range, the first compensation value and the second compensation value.

Refer to Figure 9, which is a schematic diagram for determining the reference frequency range. In step B, the reference frequency range is determined based on the user's physical condition, including:

Step B3: Determine the third compensation value of the maximum value and the fourth compensation value of the minimum value in the reference frequency range corresponding to different body states.

Step B4: Determine the reference frequency range based on the second reference frequency range, the third compensation value and the fourth compensation value.

The reference frequency range has a second reference frequency range. The second reference frequency range is used to determine the reference frequency range under different body states. Under different body states, there is a third compensation value of the maximum value and a third compensation value of the minimum value in the reference frequency range. Four compensation values. The third compensation value and the fourth compensation value corresponding to different body states are different. The reference frequency range matching the body state can be determined based on the second reference frequency range, the third compensation value and the fourth compensation value.

The first reference frequency range and the second reference frequency range may be preset. For example, the second reference frequency range is 0.3Hz to 3Hz. Different physical states will have different impacts on the respiratory frequency. The first compensation value can be and the second compensation value to dynamically adjust the target frequency range, and dynamically adjust the reference frequency range according to the third compensation value and the fourth compensation value. The first compensation value and the second compensation value may be the same, and the third compensation value and the fourth compensation value may also be the same.

For example, if the first compensation value and the second compensation value are both 0.2Hz, then the second reference frequency range is adjusted to 0.5Hz to 3.2Hz. When the first compensation value and the second compensation value are the same, the first compensation value and the second compensation value can be recorded as Delta_f, and the second reference frequency range is adjusted to Delta_f+0.3Hz to Delta_f+3Hz.

By adjusting the reference spectrum range and target frequency range corresponding to different body states, the breathing frequency in the current body state can be determined according to the body state, improving the accuracy of the breathing frequency.

In another embodiment, the method further includes:

When it is determined that the breathing frequency is abnormal, prompt information is sent to a preset device that has established a communication connection with the headset. The preset device may be a terminal device held by the user whose respiratory frequency is being monitored, such as a mobile phone, tablet computer, or other device, or may be a device held by a user who has a social relationship with the user whose respiratory frequency is being monitored.

When it is determined that the breathing frequency of the user whose breathing is being monitored is abnormal, that is, the breathing frequency exceeds the reference frequency range, a prompt message is sent to the preset device connected to the headset to notify the user or the user's associated device. personnel, thereby helping the user or the personnel associated with the user to understand the physical condition of the user whose breathing is being monitored, so as to facilitate timely medical treatment.

The prompt information can be a pop-up message, a sound prompt message or a short message, etc.

In another embodiment, refer to FIG. 10 , which is a schematic diagram of a respiratory monitoring device, which can be applied to an earphone, and the earphone includes a feedback microphone. The device includes:

The ear canal audio signal collection module 1 is configured to collect the audio signal in the ear canal through the feedback microphone to obtain the respiratory audio signal;

The respiratory audio signal determination module 2 is configured to perform filtering processing on the ear canal audio signal to obtain a respiratory audio signal; wherein the respiratory audio signal is when the earphone is worn by the user. The vibrations generated during breathing are transmitted to the ear canal through bone conduction to generate audio signals;

The respiratory frequency determination module 3 is configured to determine the respiratory frequency of the user according to the respiratory audio signal;

The abnormality determination module 4 is configured to determine whether the respiratory frequency is abnormal according to the respiratory frequency and a reference frequency range.

In another embodiment, the respiratory frequency determination module includes:

a spectrum information determining unit configured to determine the spectrum information of the respiratory audio signal;

A target amplitude determination unit configured to detect a target amplitude in the respiratory audio signal within a target frequency range based on the spectrum information; wherein the target amplitude is the maximum value of the fundamental wave of the respiratory audio signal. peak value;

The respiratory frequency determination unit is configured to determine the respiratory frequency according to the frequency corresponding to the target amplitude.

In another embodiment, the target amplitude determination unit includes:

a peak detection subunit configured to detect the maximum peak value within the target frequency range based on the spectrum information;

The waveform detection subunit is configured to determine whether there is a waveform of the second waveform frequency within the target frequency range according to the first waveform frequency of the waveform corresponding to the maximum peak value; wherein the first waveform frequency is the An integer multiple of the second waveform frequency;

The fundamental wave determination subunit is configured to determine the waveform of the second waveform frequency as the fundamental wave when it is determined that there is a waveform of the second waveform frequency; when it is determined that there is no waveform of the second waveform frequency. When the waveform is a waveform, the waveform of the first waveform frequency is determined as the fundamental wave.

In another embodiment, the respiratory frequency determining unit includes:

A first respiratory frequency determination subunit configured to determine the frequency corresponding to the target amplitude as the respiratory frequency when the number of frames of the respiratory audio signal is one frame;

The second respiratory frequency determination subunit is configured to determine the average frequency corresponding to the target amplitude in the multiple frames of the respiratory audio signal as the respiratory frequency when the number of frames of the respiratory audio signal is multiple frames. .

In another embodiment, the device further includes:

A body state determination module configured to obtain the body state of the user; the body state includes a motion state and a non-motion state, the motion state includes at least one state corresponding to motion, and the non-motion state includes at least one The state corresponding to non-motion;

a frequency range determination module configured to determine the target frequency range and the reference frequency range according to the body state;

Wherein, different body states correspond to respective target frequency ranges, and different body states respectively correspond to respective reference frequency ranges; in the same body state, the reference frequency range is included in the target frequency range. Inside.

In another embodiment, the frequency range determination module includes:

A first determination unit configured to determine a first compensation value of the maximum value and a second compensation value of the minimum value in the target frequency range corresponding to different body states;

A second determination unit configured to determine a third compensation value of the maximum value and a fourth compensation value of the minimum value in the reference frequency range corresponding to different body states;

a target frequency range determination unit configured to determine the target frequency range according to the first reference frequency range, the first compensation value and the second compensation value;

A reference frequency range determining unit is configured to determine the reference frequency range according to the second reference frequency range, the third compensation value and the fourth compensation value.

In another embodiment, the respiratory audio signal determination module includes:

A framing unit configured to frame the ear canal audio signal to obtain the ear canal audio signal after multiple frames of framing;

The filtering unit is configured to filter the ear canal audio signal obtained after multiple frames are divided into frames to obtain the respiratory audio signal of multiple frames.

In another embodiment, the device further includes:

a prompt information sending module configured to send prompt information to the preset device when it is determined that the respiratory frequency is abnormal;

Wherein, a communication connection is established between the preset device and the earphone.

In another embodiment, an embodiment of the present disclosure provides an earphone, which includes a housing and a controller, a feedback microphone, a feedforward microphone and a speaker provided on the housing;

The feedforward microphone is connected to the controller and used to collect audio data outside the ear canal and send it to the controller;

The feedback microphone is connected to the controller and used to collect audio data in the ear canal and send it to the controller;

The controller includes a memory and a processor. The memory has executable computer instructions stored therein. The processor can call the computer instructions stored in the memory to execute the method described in any embodiment.

In another embodiment, a computer storage medium is provided, and the computer storage medium stores an executable program; after the executable program is executed by a processor, the method provided in any of the above embodiments can be implemented.

FIG. 11 is a block diagram of an electronic device 800 according to an exemplary embodiment.

Referring to FIG. 11 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power supply component 806 , a multimedia component 808 , an audio component 810 , an input/output (I/O) interface 812 , and a sensor component 814 , and communication component 816.

Processing component 802 generally controls the overall operations of electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to generate all or part of the steps of the methods described above. Additionally, processing component 802 may include one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

Memory 804 is configured to store various types of data to support operations at electronic device 800 . Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EEPROM), Programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

Power supply component 806 provides power to various components of electronic device 800 . Power supply components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800 .

Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide action. In some embodiments, multimedia component 808 includes a front-facing camera and/or a rear-facing camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front-facing camera and rear-facing camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

Audio component 810 is configured to output and/or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in operating modes, such as call mode, recording mode, and voice recognition mode. The received audio signal may be further stored in memory 804 or sent via communication component 816 . In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, a button, etc. These buttons may include, but are not limited to: Home button, Volume buttons, Start button, and Lock button.

Sensor component 814 includes one or more sensors for providing various aspects of status assessment for electronic device 800 . For example, the sensor component 814 can detect the open/closed state of the device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor component 814 can also detect the electronic device 800 or a component of the electronic device 800. changes in position, the presence or absence of user contact with the electronic device 800 , the orientation or acceleration/deceleration of the electronic device 800 and changes in the temperature of the electronic device 800 . Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 4G or 5G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communications component 816 also includes a near field communications (NFC) module to facilitate short-range communications. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, electronic device 800 may be configured by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmable gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation is used to perform the above method.

In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory 804 including instructions, executable by the processor 820 of the electronic device 800 to generate the above method is also provided. For example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

Other embodiments of the invention will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common common sense or customary technical means in the technical field that are not disclosed in the present disclosure. . It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

It is to be understood that the present invention is not limited to the precise construction described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims

A breathing monitoring method, applied to earphones, the earphones including feedback microphones, the method includes:

Collect the audio signal in the ear canal through the feedback microphone to obtain the ear canal audio signal;

The ear canal audio signal is filtered to obtain a respiratory audio signal; wherein the respiratory audio signal is the vibration generated by the user during the breathing process when the earphone is worn by the user through bone conduction. The audio signal produced by being delivered to the ear canal;

Determine the user's respiratory frequency based on the respiratory audio signal;

According to the respiratory frequency and the reference frequency range, it is determined whether the user's respiratory frequency is abnormal.
The method of claim 1, wherein determining the user's breathing frequency based on the breathing audio signal includes:

Determine the spectrum information of the respiratory audio signal;

Based on the spectrum information, detect the target amplitude in the respiratory audio signal within the target frequency range; wherein the target amplitude is the maximum peak value of the fundamental wave of the respiratory audio signal;

The respiratory frequency is determined according to the frequency corresponding to the target amplitude.
The method according to claim 2, wherein detecting the target amplitude in the respiratory audio signal within the target frequency range based on the spectrum information includes:

Based on the spectrum information, detect the maximum peak value within the target frequency range;

According to the first waveform frequency of the waveform corresponding to the maximum peak value, determine whether there is a waveform of the second waveform frequency within the target frequency range; wherein the first waveform frequency is an integer multiple of the second waveform frequency;

When it is determined that the waveform of the second waveform frequency exists, the waveform of the second waveform frequency is determined as the fundamental wave;

When it is determined that there is no waveform of the second waveform frequency, the waveform of the first waveform frequency is determined as the fundamental wave.
The method according to claim 2, wherein determining the respiratory frequency according to the frequency corresponding to the target amplitude includes:

When the number of frames of the respiratory audio signal is one frame, determine the frequency corresponding to the target amplitude as the respiratory frequency;

When the number of frames of the respiratory audio signal is multiple frames, the average frequency of the frequencies corresponding to the target amplitudes in the multiple frames of the respiratory audio signal is determined as the respiratory frequency.
The method of claim 2, further comprising:

Obtain the physical state of the user; the physical state includes a moving state and a non-moving state, the moving state includes at least one state corresponding to movement, and the non-moving state includes at least one state corresponding to non-moving;

Determine the target frequency range and the reference frequency range according to the physical state;

Wherein, different body states correspond to respective target frequency ranges, and different body states respectively correspond to respective reference frequency ranges; in the same body state, the reference frequency range is included in the target frequency range. Inside.
The method according to claim 5, wherein determining the target frequency range and the reference frequency range according to the user's physical state includes:

Determine the first compensation value of the maximum value and the second compensation value of the minimum value in the target frequency range corresponding to different physical states;

Determine the third compensation value of the maximum value and the fourth compensation value of the minimum value in the reference frequency range corresponding to different body states;

Determine the target frequency range according to the first reference frequency range, the first compensation value and the second compensation value;

The reference frequency range is determined according to the second reference frequency range, the third compensation value and the fourth compensation value.
The method according to claim 1, wherein filtering the ear canal audio signal to obtain a respiratory audio signal includes:

Framing the ear canal audio signal to obtain the ear canal audio signal divided into multiple frames;

The ear canal audio signal obtained after dividing multiple frames is filtered to obtain the respiratory audio signal of multiple frames.
The method of claim 1, further comprising:

When it is determined that the respiratory frequency is abnormal, send prompt information to the preset device;

Wherein, a communication connection is established between the preset device and the earphone.
A respiratory monitoring device applied to an earphone, the earphone including a feedback microphone; the device includes:

An ear canal audio signal collection module is configured to collect the audio signal in the ear canal through the feedback microphone to obtain the ear canal audio signal;

The respiratory audio signal determination module is configured to perform filtering processing on the ear canal audio signal to obtain a respiratory audio signal; wherein the respiratory audio signal is when the earphone is worn by the user and the user is breathing. The vibrations generated during the process are transmitted to the ear canal through bone conduction to generate audio signals;

a respiratory frequency determination module configured to determine the respiratory frequency of the user based on the respiratory audio signal;

An abnormality determination module is configured to determine whether the user's breathing frequency is abnormal according to the breathing frequency and a reference frequency range.
The device of claim 9, wherein the respiratory rate determination module includes:

a spectrum information determining unit configured to determine the spectrum information of the respiratory audio signal;

A target amplitude determination unit configured to detect a target amplitude in the respiratory audio signal within a target frequency range based on the spectrum information; wherein the target amplitude is the maximum value of the fundamental wave of the respiratory audio signal. peak value;

The respiratory frequency determination unit is configured to determine the respiratory frequency according to the frequency corresponding to the target amplitude.
The device according to claim 10, wherein the target amplitude determining unit includes:

a peak detection subunit configured to detect the maximum peak value within the target frequency range based on the spectrum information;

The waveform detection subunit is configured to determine whether there is a waveform of the second waveform frequency within the target frequency range according to the first waveform frequency of the waveform corresponding to the maximum peak value; wherein the first waveform frequency is the An integer multiple of the second waveform frequency;

The fundamental wave determination subunit is configured to determine the waveform of the second waveform frequency as the fundamental wave when it is determined that there is a waveform of the second waveform frequency; when it is determined that there is no waveform of the second waveform frequency. When the waveform is a waveform, the waveform of the first waveform frequency is determined as the fundamental wave.
The device according to claim 10, wherein the respiratory frequency determining unit includes:

A first respiratory frequency determination subunit configured to determine the frequency corresponding to the target amplitude as the respiratory frequency when the number of frames of the respiratory audio signal is one frame;

The second respiratory frequency determination subunit is configured to determine the average frequency corresponding to the target amplitude in the multiple frames of the respiratory audio signal as the respiratory frequency when the number of frames of the respiratory audio signal is multiple frames. .
The device of claim 10, wherein the device further comprises:

A body state determination module configured to obtain the body state of the user; the body state includes a motion state and a non-motion state, the motion state includes at least one state corresponding to motion, and the non-motion state includes at least one The state corresponding to non-motion;

a frequency range determination module configured to determine the target frequency range and the reference frequency range according to the body state;

Wherein, different body states correspond to respective target frequency ranges, and different body states respectively correspond to respective reference frequency ranges; in the same body state, the reference frequency range is included in the target frequency range. Inside.
The device of claim 13, wherein the frequency range determination module includes:

A first determination unit configured to determine a first compensation value of the maximum value and a second compensation value of the minimum value in the target frequency range corresponding to different body states;

A second determination unit configured to determine a third compensation value of the maximum value and a fourth compensation value of the minimum value in the reference frequency range corresponding to different body states;

a target frequency range determination unit configured to determine the target frequency range according to the first reference frequency range, the first compensation value and the second compensation value;

A reference frequency range determining unit is configured to determine the reference frequency range according to the second reference frequency range, the third compensation value and the fourth compensation value.
The device of claim 9, wherein the respiratory audio signal determination module includes:

A framing unit configured to frame the ear canal audio signal to obtain the ear canal audio signal after multiple frames of framing;

The filtering unit is configured to filter the ear canal audio signal obtained after multiple frames are divided into frames to obtain the respiratory audio signal of multiple frames.
The device of claim 9, further comprising:

a prompt information sending module configured to send prompt information to the preset device when it is determined that the respiratory frequency is abnormal;

Wherein, a communication connection is established between the preset device and the earphone.
An earphone, which includes a housing and a controller, a feedback microphone, a feedforward microphone and a speaker provided on the housing;

The feedforward microphone is connected to the controller and used to collect audio data outside the ear canal and send it to the controller;

The feedback microphone is connected to the controller and used to collect audio data in the ear canal and send it to the controller;

The controller includes a memory and a processor, executable computer instructions are stored on the memory, and the processor can call the computer instructions stored on the memory to execute any one of claims 1 to 8. Methods.
A computer storage medium that stores an executable program; after the executable program is executed by a processor, the method as provided in any one of claims 1 to 8 can be implemented.