KR102108630B1 - Method for respiration measurement using ultrasonic wave based on image segmentation and device for the same - Google Patents

Abstract

According to an embodiment of the present invention, an ultrasound transceiver configured to transmit an ultrasound signal and receive a reflected ultrasound signal from a user, related to a correlation between the ultrasound signal transmitted from the ultrasound transceiver and the ultrasound signal reflected and received. Breathing including an image acquiring unit configured to acquire a correlated image, an image segmentation processing unit configured to image-correlate the correlation image, and a breathing measuring unit configured to measure respiration using internal information obtained through image segmentation A measuring device can be provided.

Description

METHOD FOR RESPIRATION MEASUREMENT USING ULTRASONIC WAVE BASED ON IMAGE SEGMENTATION AND DEVICE FOR THE SAME

The present invention relates to a method for measuring breathing using ultrasound based on image segmentation and a device therefor. More specifically, it relates to a technique for determining a breathing rate (breathing rate or respiration rate) and inhalation exhalation through image segmentation processing using a convolutional neural network (CNN), which is frequently used for image recognition, among deep learning technologies.

The mobile healthcare market is expected to reach about $ 8 billion in 2018, of which the user's heart rate, breathing and sleep pattern analysis market is very important. In addition, as the mobile technology advances, the U-Health Care market with ubiquitous technology is growing, and for non-invasive or non-contact measurement systems suitable for ubiquitous environments, which can be easily measured without burden of measurement in daily life. Research is also very active.

Currently, a contact method is widely used as a method of measuring breathing or pulse. For example, a method of measuring air flow in a mask-type device, a method of attaching a sensor to the chest, a method of collecting and analyzing data from a photoplethysmogram (PPG) sensor that measures the cardiovascular response in the skin, or a smart watch ) There is an optical pulse sensor or the like embedded in a module form.

However, in the case of the contact method, a measurement method in which a sensor or a band should be worn on the chest, fingers, or other skin is burdensome and has the disadvantage of inconvenience. Accordingly, non-contact breathing measurement methods, which are inferior in performance to current contact methods but have advantages in ease of use, have been actively studied and proposed.

Republic of Korea Registered Patent Publication 10-1070389 Republic of Korea Registered Patent Publication 10-1145646

An object of the present invention is to provide a method for measuring breathing using ultrasound and an apparatus therefor based on image segmentation.

In addition, an object of the present invention is to determine and measure respiration rate and inspiration exhalation through image segmentation processing using a correlation image of an ultrasonic signal and a convolutional neural network (CNN).

In addition, it is an object of the present invention to provide a respiratory measurement device that can more efficiently measure the characteristics related to the respiratory rate and inhalation exhalation using a U net-based CNN.

The solving problems of the present invention are not limited to the above-mentioned contents, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

In one embodiment of the present invention, an apparatus for measuring respiration using ultrasound, comprising: an ultrasound transceiver configured to transmit ultrasound signals and receive ultrasound signals reflected from a user; An image acquiring unit configured to acquire a correlation image related to correlation between the ultrasonic signal transmitted from the ultrasonic transceiver and the ultrasonic signal reflected and received; An image segmentation processing unit configured to process the correlation image by image segmentation; And a respiratory measuring unit configured to measure breathing using internal information obtained through the image segmentation.

Here, the image segmentation may use a convolutional neural network (CNN).

Also, the respiration measurement unit may include a respiration rate measurement unit for measuring a respiration rate by using the internal information and external information related to a previously-known expected range, and an inhalation exhalation determination unit for determining an inhalation exhalation.

Further, the internal information may include the slope and intensity of at least some segments due to the image segmentation.

In addition, the image segmentation processing unit is learned by using a value of a loss function, and the loss function may be based on a value related to image segmentation and a value related to inhalation determination.

In another embodiment of the present invention, a method for measuring respiration using ultrasound, comprising: receiving an ultrasound signal reflected from a user in an ultrasound transceiver; Obtaining, by the image acquisition unit, a correlation image related to a correlation between the ultrasonic signal transmitted from the ultrasonic transceiver and the ultrasonic signal reflected and received; An image segmentation process of the correlation image by the image segmentation processing unit; By the respiratory measurement unit, it is possible to provide a respiratory measurement method comprising the step of measuring respiration using internal information obtained through the image segmentation.

Here, the image segmentation may use a convolutional neural network (CNN).

In addition, the step of measuring the respiration may include measuring a respiration rate using the internal information and external information related to a previously known expected range.

Further, the internal information may include the slope and intensity of at least some segments due to the image segmentation.

In addition, the image segmentation processing unit is learned by using a value of a loss function, and the loss function may be based on a value related to image segmentation and a value related to inhalation determination.

According to the present invention, it is possible to provide a method for measuring breathing using ultrasound and an apparatus therefor based on image segmentation.

In addition, according to the present invention, it is possible to determine and measure respiration rate and inspiration exhalation through image segmentation processing using a correlation image of an ultrasound signal and a convolutional neural network (CNN).

In addition, according to the present invention, it is possible to provide a respiratory measuring apparatus capable of more efficiently measuring characteristics related to respiratory rate and exhalation by using U net-based CNN.

The effects of the present invention are not limited to the contents mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a correlation image of an ultrasonic transmission signal and an ultrasonic reception signal of an ultrasonic transceiver.
Figure 2 is a conceptual diagram for explaining the configuration of a respiratory measuring apparatus using ultrasound according to an embodiment of the present invention.
3 is a flowchart illustrating a method of measuring breathing using ultrasound according to an embodiment of the present invention.
4 is a flowchart illustrating a method of measuring breathing using ultrasound according to an embodiment of the present invention.
5A is a conceptual diagram for explaining the architecture and operation of u net, and FIG. 5b is a diagram showing a medical image segmented by u net.
6 is a diagram showing the results of image segmentation according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein.

The terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, the singular form also includes the plural form unless otherwise specified in the phrase.

As used herein, “comprises,” “comprising,” refers to the components, steps, operations, and / or elements mentioned, or the presence of one or more other components, steps, operations, and / or elements, or Addition is not excluded.

Further, terms including ordinal numbers such as first and second used in the present invention may be used to describe elements, but the elements should not be limited by terms. These terms are only used to distinguish one component from other components. In addition, in the description of the present invention, when it is determined that detailed descriptions of related known technologies may obscure the subject matter of the present invention, detailed descriptions thereof will be omitted.

In addition, the components shown in the embodiments of the present invention are shown independently to indicate different characteristic functions, and do not mean that each component is composed of separate hardware or one software component. That is, for convenience of description, each component is listed as each component, and at least two components of each component may be combined to form one component, or one component may be divided into a plurality of components to perform a function. The integrated and separated embodiments of each of these components are also included in the scope of the present invention without departing from the essence of the present invention.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. The configuration of the present invention and the effect of the action will be clearly understood through the following detailed description.

1 is a correlation image of an ultrasonic transmission signal and an ultrasonic reception signal of an ultrasonic transceiver.

The ultrasonic transceiver may include a transmitter for transmitting ultrasonic signals and a receiver for receiving reflected waves from which the transmitted signal is reflected from a user. As described above, a correlation image as shown in FIG. 1 may be obtained based on information related to correlation between the ultrasonic signal transmitted from the transmitter and the ultrasonic signal received from the receiver.

In FIG. 1, the horizontal axis represents time (t), and the vertical axis represents the delay time (τ) between the transmission signal and the reception signal. As in the correlation image shown in FIG. 1, a correlation with respect to the value of the ultrasonic signal input / output from the transceiver can be illustrated, and in addition to the correlation pattern due to the user's breath, various correlation patterns due to noise are also provided. Since it may appear, it is not easy to find a correlation pattern by the user's breath.

Therefore, in order to more efficiently and accurately find the breathing pattern of the breathing rate, the breathing cycle, and the inhalation, the motion of the body is grasped using the image segmentation technique on the correlation image as shown in FIG. It will be measurable.

Figure 2 is a conceptual diagram for explaining the configuration of a respiratory measuring apparatus using ultrasound according to an embodiment of the present invention.

The apparatus 100 for measuring breathing using ultrasound according to an embodiment of the present invention may include an ultrasound transceiver 110, an image acquiring unit 120, an image segmentation processing unit 130, and a breathing measuring unit 140.

The ultrasonic transceiver 110 may be configured to generate and transmit ultrasonic signals, and to receive ultrasonic signals reflected from a user.

The image acquiring unit 120 may be configured to acquire a correlation image related to correlation between the ultrasonic signal transmitted from the ultrasonic transceiver and the ultrasonic signal reflected and received. As described above, the correlation image generated by the image acquisition unit 120 may be generated based on a time axis and a delay time as shown in FIG. 1, and a breathing pattern may be obtained from the correlation image. This is a delay from the time of transmission to the time of arrival of the reflected signal, as the reciprocation distance of the ultrasonic signal becomes shorter due to the abdomen when exhaling, for example, when the person breathes, and the reciprocation distance of the ultrasonic signal increases as the belly enters upon exhalation. This is because periodic pattern changes in time occur.

The image segmentation processing unit 130 may be configured to process the obtained correlation image by image segmentation. Here, the image segmentation may use a convolutional neural network (CNN) applicable to image analysis among deep learning technologies. In addition, among the CNN deep learning algorithms, a u net suitable for image analysis having a complex pattern such as a medical image may be used. A more detailed description of the configuration and effect of u net will be described later with reference to FIG. 5.

Through the image segmentation processing of the image segmentation processing unit 130, the slope and intensity values of the segment pieces related to the breathing pattern may be measured. As described above, the slope and intensity information of the segment obtained by the image segmentation process from the correlation image may be referred to as internal information.

The image segmentation processing unit 130 is trained to minimize the value of the loss function, and the loss function may be based on a value set in relation to the image segmentation, thereby learning the image segmentation.

The respiration measurement unit 140 measures the respiration rate using the internal information such as the slope and intensity of the segment obtained through the image segmentation in the image segmentation processing unit 130, and thereby measures the respiration pattern of inhalation and exhalation It can be configured to.

In addition, the respiration measurement unit 140 is external information related to the above internal information and a known and predicted range, for example, information related to the respiratory rate and period of the general public, information related to the pattern of inhalation and exhalation, information related to the user's respiratory pattern, and the like. You can measure breathing more accurately and efficiently.

For example, the respiratory measurement unit 140 may include a respiratory rate measurement unit 141 for measuring a respiratory rate using internal information and external degree, and an inhalation exhalation determination unit 142 for determining an inhalation exhalation.

As another example, the image segmentation processing unit 130 may be trained to determine inhalation and exhalation based on this as well as image segmentation processing, so that an end-to-end learning method may be utilized. In this case, the loss function of the image segmentation processing unit 130 is based on a value related to the image segmentation and a value related to the inhalation exhalation determination, for example, a value related to the loss of the image segmentation determination and an inhalation exhalation determination. You can learn to minimize the sum of the values associated with loss.

According to an embodiment of the present invention, through this configuration, the correlation rate of the ultrasound signal and the image segmentation process using a convolutional neural network (CNN) are used to more effectively automate respiration rate and inhalation exhalation to determine and measure can do. In addition, it is possible to provide a respiratory measuring apparatus capable of more efficiently measuring respiratory rate and inspiration exhalation characteristics using a U net-based CNN suitable for medical image analysis.

3 is a flowchart illustrating a method of measuring breathing using ultrasound according to an embodiment of the present invention.

A method of measuring breathing using ultrasound according to an embodiment of the present invention includes receiving an ultrasound signal reflected from a user at the ultrasound transceiver 110 (S310).

Next, by the image acquisition unit 120, it is possible to obtain a correlation image related to correlation between the ultrasonic signal transmitted from the ultrasonic transceiver and the received ultrasonic signal. (S320)

Next, the image segmentation processing of the correlation image obtained by the image acquisition unit 120 may be performed by the image segmentation processing unit 130. (S330) Here, the image segmentation is CNN (Convolutional Neural Network). ), An image analysis technique using u net suitable for complex medical image analysis may be used.

Next, the respiratory measurement unit 140 may use internal information and previously known external information obtained through image segmentation (S340), through which the user's respiratory rate and cycle may be measured. (S350) , The internal information includes the slope and intensity of at least some segments by image segmentation, and the external information may include information related to a user's breathing speed and period, and information related to a pattern of exhalation and exhalation. .

In addition, it is possible to measure the breathing pattern by determining inhalation, exhalation, etc. in relation to the user's breathing. (S360)

4 is a flowchart illustrating a method of measuring breathing using ultrasound according to an embodiment of the present invention.

The method for measuring respiration using ultrasound according to an embodiment of the present invention includes receiving an ultrasound signal reflected from a user at the ultrasound transceiver 110 (S410).

Next, by the image acquisition unit 120, a correlation image related to correlation between the ultrasound signal transmitted from the ultrasound transceiver and the received ultrasound signal may be obtained (S420).

Next, the image segmentation processing of the correlation image obtained by the image acquisition unit 120 may be performed by the image segmentation processing unit 130. (S430) Here, the image segmentation is CNN (Convolutional Neural Network). ), You can use an image analysis technique using u net.

In addition, the image segmentation processing unit 130 may be trained in an end-to-end learning method by learning not only image segmentation processing but also learning to determine inhalation and exhalation based thereon. In this case, the loss function of the image segmentation processing unit 130 may be learned to minimize, for example, the value of the loss function based on a value related to image segmentation and a value related to inhalation determination.

According to the learning result, it is possible to measure the breathing pattern by determining inhalation and exhalation in relation to the user's breathing. (S440)

5A is a conceptual diagram for explaining the architecture and operation of u net.

There are various methods related to image segmentation, but u nets suitable for medical images having complex patterns can be used.

Referring to FIG. 5A, the Unet architecture includes conv (convolution), one of the activation functions, ReLU (Rectified linear unit), max pool (max pooling), and up-conv (up convolution). Up Convolution), and it can be composed of an encoder part that combines them to create detailed features and a decoder part that creates images by combining features.

5B is a view showing a medical image segmented by u net.

The U-net having the above structure is frequently used in the analysis of medical images, and FIG. 6B shows the medical image segmented into the U-net, and thus can have a good effect on the segmentation processing of the medical images having such a complex pattern.

6 is a diagram showing the results of image segmentation according to an embodiment of the present invention.

Referring to FIG. 6, the patterns of the inhalation 610 and the exhalation 620 are clearly divided, and for example, the inclination and intensity of the segment can be measured in the segment 611 where the inclination occurs. In addition, it is possible to automatically determine the breathing pattern by determining the inhalation exhalation by learning the inhalation exhalation determination.

According to the prior art, it is necessary to manually find the breathing feature manually in the correlation image as shown in FIG. 1. However, if the image segmentation process using CNN is used as in the present invention, it is possible to automatically and quickly find features related to such a breathing rate and pattern.

The embodiments disclosed in the specification of the invention are only examples, and the present invention is not limited thereto. The scope of the present invention should be interpreted by the following claims, and all technologies within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

100: ultrasonic breathing meter
110: ultrasonic transceiver
120: image acquisition unit
130: image segmentation processing unit
140: breath measurement unit
141: respiratory rate measuring unit
142: exhalation exhalation judgment

Claims (10)

In the respiratory measuring apparatus using ultrasound,
An ultrasonic transceiver configured to transmit ultrasonic signals and receive ultrasonic signals reflected from a user;
An image acquiring unit configured to acquire a correlation image related to correlation between the ultrasonic signal transmitted from the ultrasonic transceiver and the ultrasonic signal reflected and received;
An image segmentation processing unit configured to process the correlation image by image segmentation; And
It includes a breath measuring unit configured to measure the breath using the internal information obtained through the image segmentation,
The respiration measurement unit includes a respiration rate measurement unit for measuring a respiration rate using the internal information and external information related to a predetermined expected range, and an inhalation exhalation determination unit for determining inhalation exhalation. Device.
The respiratory measurement apparatus of claim 1, wherein the image segmentation uses a convolutional neural network (CNN).
delete The respiratory measurement apparatus of claim 1, wherein the internal information includes a slope and an intensity of at least some segments by the image segmentation.
The apparatus of claim 1, wherein the image segmentation processing unit is learned using a value of a loss function, and the loss function is based on a value related to image segmentation and a value related to inhalation determination. .
In the respiratory measurement method using ultrasound,
Receiving an ultrasonic signal reflected from a user at the ultrasonic transceiver;
Obtaining, by the image acquisition unit, a correlation image related to a correlation between the ultrasonic signal transmitted from the ultrasonic transceiver and the ultrasonic signal reflected and received;
An image segmentation process of the correlation image by the image segmentation processing unit; And
And measuring, by a breathing measurement unit, breathing using internal information obtained through the image segmentation.
The measuring of respiration includes measuring a respiration rate by using the internal information and external information related to a previously known expected range.
The method of claim 6, wherein the image segmentation uses a convolutional neural network (CNN).
delete The method of claim 6, wherein the internal information includes slope and intensity of at least some segments due to the image segmentation.
The method of claim 6, wherein the image segmentation processing unit is learned using a value of a loss function, and the loss function is based on a value related to image segmentation and a value related to inhalation and exhalation determination. .

Images