KR20210142386A - Apparatus and method for measuring of respiratory variation based-on image - Google Patents

Abstract

The present invention relates to an image-based breathing capacity measurement apparatus and a method thereof and, more specifically, to a technology for measuring a patient's breathing capacity in a non-contact state to understand the patient's condition and track the location of a tumor in real-time according to a lung function test or respiration. According to the present invention, an image-based breathing capacity measurement apparatus measures a user's lung capacity by using the user's breathing capacity. The apparatus comprises: a camera unit obtaining image information by photographing a user's breathing area with a plurality of cameras to which an infrared thermal imaging method is applied; a stereo unit reconstructing the image information obtained from the camera unit to generate a three-dimensional (3D) image; and a control unit extracting the infrared spectrum of the 3D image to calculate the user's lung capacity through the volume of the breathing area. Accordingly, since inspection is performed without direct contact with a user, the apparatus eliminates the user's inconvenience and provides an advantage of accurately measuring breathing capacity by using 3D stereo vision.

Description

Image-based respiratory volume measurement device and method

The present invention relates to an image-based respiratory volume measuring apparatus and method, and more particularly, to a technology for measuring a patient's respiratory volume in a non-contact state to determine the patient's condition to track the location of a tumor in real time according to a lung function test or respiration .

A diagnostic spirometer is a medical device for diagnosing the lung function of a subject by measuring the amount of gas (respiration parameters) coming out or entering the lungs of a subject. The diagnostic spirometer measures the spirometry by electronically integrating the output by connecting an airflow transducer to the patient's mouth after closing the patient's nose in order to examine the lung function.

In the related art, the diagnostic variables that can be calculated by integrating the output are Forced Vital Capacity (FVC), Forced Expiratory Volume in 1sec (FEV), Peak Expiratory Flow (PEF) Air Flow rate), Forced mid-Expiratory Air Flow rate (FET), and per second rate (FEV, FEV/FVC).

However, when the conventional technology is applied, if the lung function is continuously observed for a respiratory patient or the location of a tumor located in the lung or liver is tracked in real time through the respiration rate during radiation therapy, the patient is sealed with the mouth or nose. Since the respiration is measured, there is a problem that may cause physical and mental discomfort to the user.

An object of the present invention is to provide an image-based respiratory volume measurement apparatus and method for examining a user's lung function or tracking a change in lung volume in a non-contact method.

In order to achieve the above object, the present invention provides a first camera, a second camera or a second camera to which an infrared thermal imaging method is applied to the breathing area of the user in an image-based respiratory volume measuring device for measuring lung capacity using a user's respiratory volume. a camera unit for acquiring image information by photographing with a plurality of cameras, such as 3 cameras; a stereo unit for reconstructing the image information obtained from the camera unit to generate a 3D image; and a control unit for extracting the infrared spectrum of the three-dimensional image and calculating the lung capacity of the user based on the volume of the breathing region.

According to an embodiment, the camera unit may convert an infrared image in the image information of the user's inhalation or exhalation into a temperature image.

According to an embodiment, the camera unit may generate 3D image information by tracking a flow in which the distribution of the injected fluorescent or phosphorescent material changes.

According to an embodiment, the stereo unit may generate a three-dimensional image of the breathing area by arranging the image information captured by the plurality of cameras in the same time period in a three-dimensional space.

According to an embodiment, the controller may calculate the lung capacity by analyzing the spatial coordinates of the 3D image to calculate the volume of the breathing area.

In addition, the present invention, in the image-based respiratory volume measurement method for measuring the lung capacity using the user's respiratory volume, the image information by photographing the user's breathing area in a camera unit having a plurality of cameras to which the infrared thermal imaging method is applied obtaining; generating a 3D image in a stereo unit by reconstructing the image information; and calculating the lung capacity of the user by calculating the volume of the breathing area from the three-dimensional image generated by the stereo unit.

According to an embodiment, the obtaining of the image information may further include converting an infrared image in the image information of the user's inhalation or exhalation into a temperature image.

According to an embodiment, the method may further include generating a three-dimensional image of the breathing area by arranging the image information captured by the plurality of cameras in the same time period in a three-dimensional space.

According to the present invention having the configuration as described above, there is an advantage in that the user's inconvenience can be eliminated because the inspection is performed without direct contact with the user.

In addition, the present invention has the advantage of being able to accurately measure the volume of respiration by using three-dimensional stereo vision.

1 is a diagram for explaining a stereo vision system.
2 is a diagram illustrating a method of installing a camera in a stereo vision system.
3 is a diagram illustrating a sequence in which an image is captured by a camera in a general stereo vision system.
4 shows a state in which a camera unit is disposed in a user's breathing area to obtain image information on exhalation or inhalation according to an embodiment of the present invention.
5 shows a state in which a 3D image is generated by a stereo unit according to an embodiment of the present invention.
6 is a diagram illustrating a state in which the control unit measures a user's lung capacity according to an embodiment of the present invention.
7 is a view showing a state of injection of a fluorescent or phosphorescent material and a state of acquiring a flow thereof as a three-dimensional image according to an embodiment of the present invention.

Terms used in this specification will be briefly described, and the present invention will be described in detail.

The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

When a part "includes" a certain element throughout the specification, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software. . Also, throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being connected "with another configuration in between".

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily carry out the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

1 to 3 illustrate a general state of generating a three-dimensional image using image information obtained using a camera to which an infrared thermal imaging method is applied.

1 is a diagram for explaining a stereo vision system.

Referring to FIG. 1A , the reason that a person can three-dimensionally recognize an object is because the right eye and the left eye recognize different images. The retinal images obtained by both eyes are obtained by synthesizing with each other in the brain 100, which is obtained by a minute difference between the images obtained by both eyes. The image acquired by this minute difference is transmitted to the processing part of the brain through the optic nerve of the retina, and the transmitted electrical signal finds the characteristics through various processing processes of the brain, and compares it with the information stored in the brain. will recognize

Referring to FIG. 1B , a stereovision system that introduces a process in which a person recognizes an object through both eyes is introduced into computer vision. Accordingly, the stereo vision system includes a camera 110 corresponding to a human eye and a computer 120 corresponding to a human visual system.

2 is a diagram illustrating a method of installing a camera in a stereo vision system.

Referring to FIG. 2 , a method of installing a camera in a stereo vision system can be largely divided into a sequential type and a simultaneous type. The sequential method is a method of inputting images sequentially using a single camera without installing cameras at the same time.

On the other hand, the simultaneous method installs both cameras and acquires images at the same time, and is a method commonly used in stereo vision systems currently. Simultaneous mode is further divided into (a) Parallel Camera Setup and (b) Crossing Camera Setup depending on the camera configuration, which is classified according to how the optical axis of the camera is configured. The parallel expression refers to vertical or horizontal parallelism, and the intersection method is close to the structure of the human eye in a way that makes the optical axis of the camera coincide.

The parallel installation method has the advantage of easy distance measurement, but has the disadvantage that it cannot produce a high-quality stereoscopic image because it does not have a parallax control function of the stereoscopic image. It has the advantage of being able to

Stereo-vision refers to a technology capable of extracting 3D image information using flat image information having two phase differences. This is a computer re-implementation of the technique of extracting depth information that cannot be obtained from planar information by analyzing the phase difference between two plane image information obtained from both eyes of a human and two plane image information from the position difference between the two eyes in the brain. . Implementation of this technique consists of steps of obtaining flat image information using a plurality of cameras and extracting 3D position information by correcting a position difference between each camera through a standard image. By using the three-dimensional information of the image obtained through the above method, the area and volume of a specific region can be obtained, and through this, the patient's respiration volume during inspiration and exhalation can be calculated.

Thermal imaging refers to an imaging technique that uses an infrared camera to convert light emitted from an object into the temperature of the object. The fact that all objects emit blackbody radiation and the fact that the radiation wavelength at room temperature is in the infrared region means that an infrared camera is suitable as an instrument for measuring the temperature of an object at room temperature. The temperature of an object can be estimated through the spectrum of infrared radiation emitted from the object, and this corresponds to matter in all states, including solid, liquid, and gas. The use of the thermal imaging camera in the present invention is that it is possible to calculate the isothermal volume by photographing the temperature of exhalation with a thermal imaging camera by using the fact that the temperature of the air staying in the patient's lungs is different from the external temperature. If this is the volume of exhalation, the lung capacity of the patient can be calculated from the volume of air during exhalation of the patient.

3 is a diagram illustrating a sequence in which an image is captured by a camera in a general stereo vision system.

As shown in FIG. 3 , on the basis of four frames, an image grabbing in a general stereo vision system is performed at an intersection of time-synchronized images taken by two different cameras.

For example, as shown in FIG. 3 , when an image is taken with two cameras having a shooting speed of 450 fps (frame per second), an image of 450 frames per second can be acquired through each camera, and the The distance and direction of the photographed subject can be checked using the time-synchronized image.

4 shows a state in which a camera unit is disposed in a user's breathing area to obtain image information on exhalation or inhalation according to an embodiment of the present invention.

Referring to FIG. 4 , in the image-based respiratory volume measuring apparatus for measuring lung capacity using the user's respiration volume, a plurality of first, second, or third cameras to which the infrared thermal imaging method is applied to the breathing area of the user It may include a camera unit for obtaining image information by photographing with a camera of

Also, although not shown, a stereo unit for generating a 3D image by reconstructing the image information obtained from the camera unit; And extracting the infrared spectrum of the three-dimensional image may include a control unit for calculating the lung capacity of the user through the volume of the breathing area.

In general, just as the distance, which is 3D information, can be inferred from the 2D images obtained respectively due to the position difference between the human left and right eyes, the 3D distance of the points in the image can be calculated from the images obtained by the two cameras with the position difference. can

Using images captured by two cameras at different positions, points of the corresponding image may be located in a three-dimensional space.

The present invention provides an image-based respiratory volume measurement method for measuring lung capacity using a user's respiration volume, In a camera unit having a plurality of cameras such as a first camera, a second camera or a third camera to which an infrared thermal imaging method is applied acquiring image information by photographing the user's breathing area; generating a 3D image in a stereo unit by reconstructing the image information; and calculating the lung capacity of the user by calculating the volume of the breathing area from the three-dimensional image generated by the stereo unit.

In particular, the obtaining of the image information may further include converting an infrared image in the image information of the user's inhalation or exhalation into a temperature image.

5 shows a state in which a 3D image is generated by a stereo unit according to an embodiment of the present invention.

The camera unit may convert an infrared image in the image information of the user's inhalation or exhalation into a temperature image.

Referring to FIG. 5 , as described in FIGS. 1 to 3 , two or more cameras require a calibration process corresponding to a mutual position difference, and through this, each Image information in the depth direction of the points can be obtained.

5 is an embodiment of a three-dimensional image, in which the breathing region is expressed in the form of a circle, and the actual breathing region cannot be in the form of an accurate circle. The control unit may include a storage unit for storing each coordinate of the 3D image, and may measure the volume by using each coordinate in the 3D space.

The stereo unit may generate a three-dimensional image of the breathing area by arranging image information photographed by a plurality of cameras, such as the first camera, the second camera, or the third camera, in a three-dimensional space in the same time zone, The controller may calculate the lung capacity by analyzing the spatial coordinates of the 3D image to calculate the volume of the breathing area.

When the user (patient) exhales, the exhaled breath temperature is similar to body temperature, so there may be a difference from the external temperature. The temperature of exhaled breath coming out of the mouth and nose is tracked in real time with a thermal image, and the volume of exhaled breath can be inversely calculated in three dimensions from the temperature difference and heat flow image of the thermal image. In this way, it is possible to track the actual patient's lung volume, not just respiration monitoring (the patient's inspiratory and exhalation cycle).

6 is a diagram illustrating a state in which the control unit measures a user's lung capacity according to an embodiment of the present invention.

7 is a view showing a state of injection of a fluorescent or phosphorescent material and a state of acquiring a flow thereof as a three-dimensional image according to an embodiment of the present invention.

Referring to FIG. 7 , a flow in which a fluorescent or phosphorescent material is sprayed in the air, the fluorescent or phosphorescent material sprayed from the camera unit to the breathing area is photographed, and the distribution of the fluorescent or phosphorescent material is changed in the stereo unit. It is possible to generate 3D image information by tracking.

Although the present invention has been described in detail through representative embodiments above, those of ordinary skill in the art to which the present invention pertains will understand that various modifications are possible within the limits without departing from the scope of the present invention with respect to the above-described embodiments. will be. Therefore, the scope of the present invention should not be limited to the described embodiments and should be defined by all changes or modifications derived from the claims and equivalent concepts as well as the claims to be described later.

Claims (9)

In the image-based respiratory volume measurement device for measuring lung capacity using the user's respiration volume,
a camera unit for obtaining image information by photographing the user's breathing area with a plurality of cameras to which an infrared thermal imaging method is applied;
a stereo unit for reconstructing the image information obtained from the camera unit to generate a 3D image; and
and a controller for extracting the infrared spectrum of the three-dimensional image and calculating the lung capacity of the user based on the volume of the breathing area.
The method of claim 1,
The camera unit,
Image-based respiration volume measurement device, characterized in that converting the infrared image in the image information of the user's inhalation or exhalation into a temperature image.
The method of claim 1,
The stereo unit,
Image-based respiration volume measuring device, characterized in that by arranging the image information taken by the plurality of cameras in the same time period in a three-dimensional space to generate a three-dimensional image of the breathing area.
The method of claim 1,
The control unit is
Image-based respiratory volume measuring apparatus, characterized in that by analyzing the spatial coordinates of the three-dimensional image to calculate the volume of the breathing area to calculate the lung capacity.
4. The method according to claim 2 or 3,
The camera unit,
Taking a fluorescent or phosphorescent material injected into the breathing area,
The stereo unit,
Image-based respiration volume measurement device, characterized in that by tracing the flow in which the distribution of the fluorescent or phosphorescent material changes to generate three-dimensional image information.
In the image-based respiratory volume measurement method for measuring lung capacity using the user's respiratory volume,
obtaining image information by photographing the breathing area of the user in a camera unit having a plurality of cameras to which an infrared thermal imaging method is applied;
generating a 3D image in a stereo unit by reconstructing the image information; and
Image-based respiration volume measurement method comprising the step of calculating the lung capacity of the user by calculating the volume of the breathing area from the three-dimensional image generated by the stereo unit.
7. The method of claim 6,
The step of obtaining the image information includes:
Image-based respiration volume measurement method further comprising the step of converting an infrared image in the image information of the user's inhalation or exhalation into a temperature image.
7. The method of claim 6,
Image-based respiration volume measurement method further comprising the step of arranging the image information taken by the plurality of cameras in the same time period in a three-dimensional space to generate a three-dimensional image of the breathing area.
9. The method according to claim 7 or 8,
photographing the fluorescent or phosphorescent material injected into the breathing area by the camera unit; or
Image-based respiration volume measurement device, characterized in that it further comprises the step of generating three-dimensional image information by tracking the flow in which the distribution of the fluorescent or phosphorescent material changes in the stereo unit.

Images