CN114897871A

CN114897871A - Bubble detection device and bubble detection method based on light refraction and reflection

Info

Publication number: CN114897871A
Application number: CN202210613409.5A
Authority: CN
Inventors: 侯景山; 黄孝民
Original assignee: Kailian Medical Technology Shanghai Co ltd
Current assignee: Kailian Medical Technology Shanghai Co ltd
Priority date: 2022-05-23
Filing date: 2022-05-23
Publication date: 2022-08-12

Abstract

The invention discloses a light refraction and reflection-based bubble detection device and a light refraction and reflection-based bubble detection method, which comprise a light source module, a macro camera and an image processing and analyzing unit; the light source module is used for irradiating the infusion pipeline, and the light rays have refraction, reflection, focusing and scattering phenomena due to the change of the density of the propagation medium, so that the edges of the pipeline wall and the bubbles present sharp and bright boundaries; the macro camera is used for receiving software instructions to realize close-range shooting of the infusion pipeline and transmitting image data to the data analysis and processing unit; and the image processing and analyzing unit analyzes and processes the pipeline image by means of decolorizing, sharpening and edge detecting technologies, marks bubble positions, counts the number of bubbles, calculates the size of the bubbles and gives an alarm according to a set threshold value. The invention utilizes light refraction and reflection to detect bubbles, has high detection precision, simple structure, low cost, stability and reliability, and can detect not only bubbles but also micro liquid drops attached to the tube wall.

Description

Bubble detection device and bubble detection method based on light refraction and reflection

Technical Field

The invention relates to a bubble detection method and device, in particular to a bubble detection device and method based on light refraction and reflection.

Background

In the process of conveying medical liquid, air in an infusion pipeline has a great threat to life safety of a patient, and if more air bubbles enter a blood vessel, air embolism can be formed and serious complications are caused, so that the patient suffers from palpitation, chest distress, accelerated heart rate, precordial pain, blood pressure reduction and the like. In many clinical applications, a device capable of reliably detecting bubbles in an infusion pipeline is needed, and the bubbles or bubble groups in the pipeline are alarmed in time and are rapidly treated.

At present, the common methods for detecting bubbles in a transfusion pipeline mainly comprise three methods: capacitance method, photoelectric method and ultrasonic detection method.

A capacitance method: two capacitance plates are respectively arranged at two sides of the infusion tube to detect the capacitance change condition between the two plates, and the change condition of the internal medium is presumed according to the change, thereby achieving the detection purpose. The advantages and disadvantages are as follows: the structure is simple, non-contact measurement is convenient to realize, but the performance is unstable, the circuit interference is easy to occur, and the interference is difficult to eliminate.

A photoelectric method: by utilizing the photoelectric effect of a photoelectric device, the common photoelectric device comprises a phototriode, a photosensitive diode and the like, and the relation between the output voltage and the illumination intensity can be obtained according to the volt-ampere characteristic. When there is a bubble in the blood delivery tube, the intensity of light received by the photosensor changes due to reflection of light and absorption of light by different media, causing a change in output voltage. The photoelectric type has the advantages of fast response, non-contact and the like, is widely applied to the automatic monitoring and control technology, is sensitive to the color of a medium, and has no obvious difference on liquid columns and bubbles

An ultrasonic method: the ultrasound propagates in a straight line in a uniform medium, but when the ultrasound reaches an interface or different media, the ultrasound undergoes reflection and refraction, and obeys a reflection law similar to geometric optics, and a transmission method, a reflection method, a frequency method and the like are commonly used in detection technology. The ultrasonic detector has small attenuation in liquid column and solid, strong penetrating power, obvious interface reflection and refraction, and ultrasonic high-frequency characteristic, and is convenient for counting actual pulse and ultrasonic pulse to judge the size of bubble and the length of continuous liquid column. The ultrasonic air breaking detector has high sensitivity and good reliability, can detect the continuous bubbles in small gaps, but has high hardware cost and complex algorithm and needs higher calculation force support.

The three detection methods have the common defects of low detection precision and sensitivity for the tiny bubbles of the small-caliber infusion pipeline, unobvious detection effect and incapability of visual observation, and greatly restrict the clinical application of the equipment.

Disclosure of Invention

The invention aims to solve the technical problem that a bubble detection device and a bubble detection method based on light refraction and reflection are used for carrying out optical imaging on a transfusion pipeline and bubbles by utilizing refraction and reflection phenomena existing when light is transmitted in different media, and identifying and analyzing image data by utilizing a software algorithm, so that bubbles (groups) in the pipeline are found and an alarm is given out in time, the problems of international small-caliber pipeline bubble detection and infinitesimal bubble detection are solved, the effective detection of ultra-tiny bubbles in the transfusion pipeline with the inner diameter smaller than 1mm can be realized, and the detection precision and the sensitivity are kept not to be reduced.

The invention is realized by the following technical scheme: a bubble detection device based on light refraction and reflection comprises a light source module, a macro camera and an image processing and analyzing unit;

the light source module is used for irradiating the infusion pipeline, and when light rays are transmitted among the pipeline, the liquid medicine and the bubbles, the density of a transmission medium is changed, and the light rays have refraction, reflection, focusing and scattering phenomena, so that the edges of the pipeline wall and the bubbles present sharp and bright boundaries;

the signal output end of the macro camera is connected with the image processing and analyzing unit, and the macro camera with an automatic focusing function is used for receiving a software instruction to realize close-range shooting of the infusion pipeline and transmitting image data to the data analyzing and processing unit;

and the image processing and analyzing unit analyzes and processes the pipeline image by means of decolorizing, sharpening and edge detecting technologies, marks bubble positions, counts the number of bubbles, calculates the size of the bubbles and gives an alarm according to a set threshold value.

According to the bubble detection method based on light refraction and reflection, light emitted by a light source module irradiates the outer wall of a transfusion pipeline and is transmitted into liquid medicine, and due to the fact that air and the attributes of the pipeline wall, the pipeline wall and the liquid medicine, and the liquid medicine and a bubble medium are different, the density difference is large, and the light is refracted and reflected for many times when propagating in the pipeline;

the greater the density difference between the media, the greater the refraction angle of the light when crossing the media boundary, and the obvious brightness difference exists between the light in the two media under a specific angle, thereby forming a bright distinguishable boundary;

the method comprises the following specific steps:

s1, after the real-time picture shot by the macro camera is transmitted to the rear-end image processing and analyzing unit, the image processing unit firstly carries out the decolorizing processing on the image, the algorithm is to scan the image, and the gray level processing is carried out aiming at the RGB value of each pixel point in the image;

s2, the decolored picture needs to be further sharpened, and the sharpening is used for improving the contrast of the picture and improving the image recognition accuracy;

s3, after the image is subjected to decolorizing and sharpening, the image is further processed and identified by using an edge detection algorithm, wherein the edge processing aims at searching for the boundary between gas phase and liquid phase and improving the identification accuracy of bubbles;

s4, after the edge detection is finished, detecting the bubbles in the pipeline by two different methods according to the graphic characteristics of the large bubbles and the micro bubbles, wherein the large bubble detection method uses a radial color gamut method for identification, and the micro bubbles use a mode matching method for judgment;

s5, after the bubbles in the pipeline are identified, the image processing and analyzing unit can send out an alarm in time and automatically process according to the setting of a user, so that the risk in the liquid medicine infusion process is greatly reduced, and the manual detection intensity and the misjudgment action are reduced.

As a preferred technical solution, in S1, there are two methods for converting a color image into a grayscale image:

the first method is to make the values of the three RGB components equal, and then the gray level image can be obtained after the output;

the second method is to convert RGB into YCbCr format, extract Y component, where the Y component in the YCbCr format represents the brightness of the image, so that only the Y component is output, and the obtained image is a gray image, and the pixel is processed by using the RGB 888-to-YCrCb calculation formula:

Y＝0.299R+0.587G+0.114B

Cb＝0.568(B-Y)+128＝-0.172R-0.339G+0.511B+128

Cr＝0.713(R-Y)+128＝0.511R-0.428G-0.083B+128。

preferably, in S2, the sharpening algorithm processes the picture pixels according to the laplacian enhancement formula, where y (m, n) ═ x (m, n) + λ × z (m, n), where x (m, n) is the picture before processing, y (m, n) is after sharpening, z (m, n) represents the edge and details (high frequency part) of the enhanced image, and λ is the enhancement factor.

As a preferred technical solution, in S3, the edge detection algorithm includes Robert operator, Sobel operator, Laplace operator, right lower edge extraction algorithm, prewitt operator, Robinson operator, Kirsch operator, and smooth operator.

As a preferred technical scheme, in S4, the radial color gamut method is to intercept a small section of pipeline picture in the vertical pipeline direction for color gamut analysis, determine whether there is any bubble according to color component distribution and continuity in the color spectrum, and perform vertical scanning on pixels in a selected interval, so that when continuity of light and shade of a scanning line is interrupted, it can be determined that liquid is cut off, and the volume of the bubble is the continuous scanning cut-off width and the inner cross-sectional area of the pipeline;

a mode matching method for identifying the micro-bubbles: the method comprises the steps of scanning a rectangular area of an image by using a window with a variable size, distinguishing contents in the window by brightness and darkness to judge whether an arc formed by white pixels exists in the rectangular area, if so, determining the arc as a bubble and marking the arc, and if the length of the rectangular side is L, determining the volume size of the bubble as 4/3 pi r 3 by using a V sphere, wherein r is L/2.

As a preferable embodiment, in S4, the mode matching method for identifying the microbubbles comprises the following steps:

taking the center of a rectangle as the center of a circle, recording an inscribed circle of the rectangular area as P1, and setting the radius as R1;

step two, establishing a second circle with the center of the rectangle as the center of a circle and the radius of R2 as P2, wherein R1 is greater than R2, and the difference is about 3-4 pixels;

and step three, calculating the ratio of bright and dark pixels in the area surrounded by the P1 and the P2, and judging that bubbles exist in the area when the ratio of the bright area pixels is more than 30%.

Compared with the traditional detection method, the method has the following advantages:

1. the detection precision is high, the invention can effectively find the micro bubbles and bubble groups in the infusion pipeline or the micro bubbles adhered to the wall of the infusion pipeline, the detection precision can reach 1uL or even higher, and the detection precision is superior to a photoelectric method, a capacitance method and an ultrasonic detection method;

2. the structure is simple, the pipeline only needs to be imaged by using the microspur camera in the structure related to the pipeline, and the imaged data is calculated and analyzed by a computer, so the structure is simple;

3. the cost is low, the required components of the invention comprise a light source, a macro camera and an embedded computer which are mature components, the research and development period is short, and the software algorithm is simple;

4. the method is stable and reliable, the method is slightly influenced by external interference factors, and the demand on computing power is low;

5. not only bubbles but also fine droplets adhering to the tube wall can be detected.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a schematic diagram of a bubble detection device based on light refraction and reflection according to the present invention;

FIG. 2 is a schematic diagram of the light path formed in the pipe after multiple reflections and refractions of incident light according to the present invention;

FIG. 3 is a schematic view of a sharp boundary formed in a pipe by reflection and refraction;

FIG. 4 is a first boundary pattern formed by the ultra-micro bubbles (groups);

FIG. 5 is a second boundary pattern formed by the ultra-micro bubbles (groups);

FIG. 6 is a comparison graph of the effect of image decoloring and sharpening;

FIG. 7 is a first diagram illustrating the edge detection effect;

FIG. 8 is a second graph of edge detection effect;

FIG. 9 is a diagram illustrating the effect of identifying large bubbles by radial color gamut method;

fig. 10 is a diagram illustrating the effect of identifying the microbubbles by the pattern matching scanning method.

Detailed Description

All of the features disclosed in this specification, or all of the steps in any method or process so disclosed, may be combined in any combination, except combinations of features and/or steps that are mutually exclusive.

Any feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving equivalent or similar purposes, unless expressly stated otherwise. That is, unless expressly stated otherwise, each feature is only an example of a generic series of equivalent or similar features.

In the description of the present invention, it is to be understood that the terms "one end", "the other end", "outside", "upper", "inside", "horizontal", "coaxial", "central", "end", "length", "outer end", and the like, indicate orientations or positional relationships based on those shown in the drawings, and are used only for convenience in describing the present invention and for simplicity in description, and do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed in a particular orientation, and be operated, and thus, should not be construed as limiting the present invention.

Further, in the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.

The use of terms such as "upper," "above," "lower," "below," and the like in describing relative spatial positions herein is for the purpose of facilitating description to describe one element or feature's relationship to another element or feature as illustrated in the figures. The spatially relative positional terms may be intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as "below" or "beneath" other elements or features would then be oriented "above" the other elements or features. Thus, the exemplary term "below" can encompass both an orientation of above and below. The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly

In the present invention, unless otherwise explicitly specified or limited, the terms "disposed," "sleeved," "connected," "penetrating," "plugged," and the like are to be construed broadly, e.g., as a fixed connection, a detachable connection, or an integral part; can be mechanically or electrically connected; they may be directly connected or indirectly connected through intervening media, or they may be connected internally or in any other suitable relationship, unless expressly stated otherwise. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.

As shown in fig. 1, the bubble detection device based on light refraction and reflection of the present invention includes a light source module 1, a macro camera 2 and an image processing and analyzing unit 3;

When the device works, light rays emitted by the light source irradiate the outer wall of the infusion pipeline and are transmitted into the liquid medicine, the air and the pipe wall, the pipe wall and the liquid medicine, the liquid medicine and the bubble medium have different properties and have larger density difference, and the light rays are refracted and reflected for many times when being transmitted in the liquid medicine, as shown in figure 2.

The greater the density difference between the media, the greater the angle of refraction of a ray when crossing a media boundary, and the greater the difference in brightness of the rays in the two media at a particular angle, forming a bright distinguishable boundary, as shown in fig. 3.

For the more minute bubbles suspended in the liquid or attached to the tube wall, the curvature of the bubble surface is large due to the surface tension effect, so that the light focusing or scattering effect similar to a convex lens or a concave lens is generated, and the bubbles have obvious brightness distinction from the surrounding environment under the irradiation of light, thereby providing feasibility for optical identification, as shown in fig. 4 and fig. 5.

After the real-time picture shot by the macro camera is transmitted to the rear-end image processing and analyzing unit, the image processing unit firstly carries out the decoloring processing on the image, the algorithm is to scan the image, and the gray level processing is carried out on the RGB value of each pixel point in the image. There are two methods for converting a color image into a grayscale image:

the first method is to make the values of the three RGB components equal. After output, a gray scale image can be obtained.

The second method is to convert RGB into YCbCr format, and extract Y components, which represent the brightness of the image, so that only Y components are output and the resulting image is a gray image.

In this embodiment, the RGB888 to YCrCb calculation formula is used to process the pixel:

Y＝0.299R+0.587G+0.114B；

Cb＝0.568(B-Y)+128＝-0.172R-0.339G+0.511B+128；

Cr＝0.713(R-Y)+128＝0.511R-0.428G-0.083B+128。

the decolored picture needs to be further sharpened, the contrast of the picture is improved, the image recognition accuracy is improved, and the sharpening algorithm processes the picture pixels according to a Laplace enhancement formula.

y (m, n) ═ x (m, n) + λ × z (m, n), where x (m, n) is the picture before processing, y (m, n) is after sharpening, z (m, n) represents the edges and details (high frequency parts) of the enhanced image, and λ is the enhancement factor, as shown in fig. 6.

After the image is subjected to decolorizing and sharpening, the image is further processed and identified by using an edge detection algorithm. The purpose of edge processing is to find the boundary between the gas phase and the liquid phase and improve the identification accuracy of bubbles.

The existing mature edge detection algorithm with good detection effect comprises a Robert operator, a Sobel operator, a Laplace operator, a right lower edge extraction algorithm, a prewitt operator, a Robinson operator, a Kirsch operator and a smooth operator. In the present embodiment, the smoothened operator is used to identify the image, as shown in fig. 7 and 8.

After the edge detection is finished, aiming at the graphic characteristics of large bubbles and micro bubbles, the invention uses two different methods to detect the bubbles in the pipeline.

The large bubble detection method uses a radial color gamut method for identification, and the micro-bubbles use a mode matching method for judgment.

The radial color gamut method is to intercept a small section of pipeline picture in the direction vertical to a pipeline to perform color gamut analysis, and judge whether bubbles exist according to the distribution and continuity of color components in a color spectrum. As shown in fig. 9, by vertically scanning the pixels in the selected interval, it can be determined that the liquid is cut off when the continuity of the light and the shade of the scanning line is interrupted, and the bubble volume is the cut-off width of the continuous scanning.

The method is different from a large bubble identification and judgment method, and the micro-bubble identification is mainly carried out by using a mode matching method. As shown in fig. 8, a rectangular area of the image is scanned by using a window with a variable size, and the content in the window is distinguished by brightness and darkness to determine whether an arc formed by white pixels exists in the rectangular area. If the bubble exists, the bubble is determined and marked, and the side length of the rectangle is set as L, the volume size of the bubble is determined by using a V sphere of 4/3 pi r 3, wherein r is L/2. The method comprises the following specific steps:

and step three, calculating the ratio of bright and dark pixels in the area surrounded by the P1 and the P2, and judging that bubbles exist in the area when the ratio of the bright area pixels is more than 30%, as shown in FIG. 10.

After the bubbles existing in the pipeline are identified, the image processing and analyzing unit can send out an alarm in time and automatically handle according to user setting, so that the risk existing in the liquid medicine infusion process is greatly reduced, and the manual detection intensity and the existing misjudgment behaviors are reduced.

The above description is only an embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that are not thought of through the inventive work should be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope defined by the claims.

Claims

1. The utility model provides a bubble detection device based on light is refracted and reflected which characterized in that: the system comprises a light source module, a macro camera and an image processing and analyzing unit;

2. A bubble detection method based on light refraction and reflection is characterized in that: the bubble detection device of claim 1, wherein the light emitted by the light source module irradiates the outer wall of the infusion tube and is transmitted into the liquid medicine, and the light has a large density difference due to the different properties of air and tube wall, tube wall and liquid medicine, and liquid medicine and bubble medium, and is refracted and reflected for multiple times when propagating in the liquid medicine;

the method comprises the following specific steps:

3. A method for detecting bubbles based on light refraction and reflection according to claim 2, wherein: in S1, there are two methods for converting the color image into the grayscale image:

Y＝0.299R+0.587G+0.114B

Cb＝0.568(B-Y)+128＝-0.172R-0.339G+0.511B+128

Cr＝0.713(R-Y)+128＝0.511R-0.428G-0.083B+128。

4. a method for detecting bubbles based on light refraction and reflection according to claim 1, wherein: in S2, the sharpening algorithm processes the picture pixels according to the laplacian enhancement formula, where y (m, n) ═ x (m, n) + λ × z (m, n), where x (m, n) is the picture before processing, y (m, n) is after sharpening, z (m, n) represents the edges and details (high frequency portions) of the enhanced image, and λ is the enhancement factor.

5. A method for detecting bubbles based on light refraction and reflection according to claim 1, wherein: in S3, the edge detection algorithm includes Robert operator, Sobel operator, Laplace operator, lower right edge extraction algorithm, prewitt operator, Robinson operator, Kirsch operator, and smoothened operator.

6. A method for detecting bubbles based on light refraction and reflection according to claim 1, wherein: in the S4, the radial color gamut method is that a small segment of pipeline picture is cut in the direction of a vertical pipeline to carry out color gamut analysis, whether bubbles exist is judged according to color component distribution and continuity in a color spectrum, pixels in a selected interval are vertically scanned, when the continuity of light and shade of a scanning line is interrupted, the liquid can be judged to be cut off, and the volume of the bubbles is the continuous scanning cut-off width and the inner section area of the pipeline;

mode matching method for identifying micro-bubbles: the method comprises the steps of scanning a rectangular area of an image by using a window with a variable size, distinguishing contents in the window by brightness and darkness to judge whether an arc formed by white pixels exists in the rectangular area, if so, determining the arc as a bubble and marking the arc, and if the length of the rectangular side is L, determining the volume size of the bubble as 4/3 pi r 3 by using a V sphere, wherein r is L/2.

7. A method for detecting bubbles based on light ray refraction and reflection according to claim 1, wherein: in S4, the specific steps of the pattern matching method for identifying the fine bubbles are as follows: