CN113298089A - Venous transfusion liquid level detection method based on image processing - Google Patents
Venous transfusion liquid level detection method based on image processing Download PDFInfo
- Publication number
- CN113298089A CN113298089A CN202110535142.8A CN202110535142A CN113298089A CN 113298089 A CN113298089 A CN 113298089A CN 202110535142 A CN202110535142 A CN 202110535142A CN 113298089 A CN113298089 A CN 113298089A
- Authority
- CN
- China
- Prior art keywords
- liquid level
- infusion bottle
- training
- picture
- infusion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000007788 liquid Substances 0.000 title claims abstract description 119
- 238000001514 detection method Methods 0.000 title claims abstract description 58
- 238000012545 processing Methods 0.000 title claims abstract description 31
- 238000001802 infusion Methods 0.000 claims abstract description 59
- 238000000034 method Methods 0.000 claims abstract description 42
- 230000008859 change Effects 0.000 claims abstract description 8
- 238000012549 training Methods 0.000 claims description 33
- 238000012360 testing method Methods 0.000 claims description 10
- 239000003978 infusion fluid Substances 0.000 claims description 9
- 238000011176 pooling Methods 0.000 claims description 9
- 238000001990 intravenous administration Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 238000003708 edge detection Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 7
- 238000013527 convolutional neural network Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 4
- 230000036544 posture Effects 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000012806 monitoring device Methods 0.000 abstract description 2
- 239000000243 solution Substances 0.000 description 7
- 239000003814 drug Substances 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000013210 evaluation model Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 244000118350 Andrographis paniculata Species 0.000 description 1
- 206010019233 Headaches Diseases 0.000 description 1
- 208000002193 Pain Diseases 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 238000012377 drug delivery Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 231100000869 headache Toxicity 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000000968 medical method and process Methods 0.000 description 1
- 239000008267 milk Substances 0.000 description 1
- 210000004080 milk Anatomy 0.000 description 1
- 235000013336 milk Nutrition 0.000 description 1
- 230000000474 nursing effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 239000002352 surface water Substances 0.000 description 1
- 230000008961 swelling Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
- A61M5/00—Devices for bringing media into the body in a subcutaneous, intra-vascular or intramuscular way; Accessories therefor, e.g. filling or cleaning devices, arm-rests
- A61M5/14—Infusion devices, e.g. infusing by gravity; Blood infusion; Accessories therefor
- A61M5/168—Means for controlling media flow to the body or for metering media to the body, e.g. drip meters, counters ; Monitoring media flow to the body
- A61M5/16831—Monitoring, detecting, signalling or eliminating infusion flow anomalies
- A61M5/1684—Monitoring, detecting, signalling or eliminating infusion flow anomalies by detecting the amount of infusate remaining, e.g. signalling end of infusion
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Abstract
The invention provides a venous transfusion liquid level detection method based on image processing, which is characterized by comprising the following steps of: s1: collecting the picture of the infusion bottle during infusion by a camera; s2: processing the picture, and carrying out target detection on the infusion bottle through an SSD model; s3: detecting the liquid level of the detected target infusion bottle in the S2 by adopting a projection method, and detecting the position of the liquid level of the infusion bottle; s4: and judging whether the liquid level reaches a warning line. The venous transfusion liquid level monitoring device has the advantages that the change condition of the liquid level during venous transfusion can be known in time, and convenience can be provided for medical staff and patients.
Description
Technical Field
The invention relates to the field of image processing, in particular to a venous transfusion liquid level detection method based on image processing.
Background
The intelligent medical treatment combines technologies such as big data, cloud computing and Internet of things and is used for improving the relation among patients, medical care personnel and a medical environment. Meanwhile, the medical environment develops towards informatization and intelligent management, the service quality and efficiency in the medical process are greatly improved, and remote intelligent monitoring is realized. Intravenous infusion is a common treatment method in the medical treatment process, and has very wide application because of high drug delivery speed and good treatment effect. In medical infusion, the problem that the liquid level height of an infusion bottle cannot be checked in time is always that a patient and a nursing staff are quite headache. If the transfusion bottle is not processed in time, the transfusion bottle is empty, the blood return condition can occur, the injection part of the patient has swelling pain if the transfusion bottle is not processed in time, and the patient can be shocked if the transfusion bottle is not processed in time. When the number of patients is large and medical staff is limited, the condition of each patient cannot be considered. The existing liquid level detection method has the defects that the mechanical detection method has the problems of complex structure, low precision and the like; the non-contact measurement method is like ultrasonic wave and is expensive.
Patent document No. CN104819754B discloses a method for detecting the liquid level of a medicine bottle based on image processing. The key point of the invention is that a template matching method is used, a typical drug image is collected to create a reference template image, a matching template is established, and a matching template coordinate system is created; matching the collected images of the plurality of detected medicine bottles with the matching template one by one to obtain a displacement matrix of the images of the detected medicine bottles, and obtaining a corrected image of the images of the detected medicine bottles by affine transformation; then, dividing the corrected image to obtain a detection area and extracting the shape of the liquid level band; and then, calculating the lowest point of the liquid level based on the shape of the liquid level belt, and judging the qualification of the medicine bottle according to the relationship between the lowest point of the liquid level and the set threshold value of the highest point. The method has the disadvantage that when the difference between the template and the image to be detected is large, the detection cannot be completed well.
Patent document No. CN105181082B discloses a liquid level detection method and apparatus based on visible light and image processing. Firstly, establishing a measurement coordinate system with the background of the inner wall of a container for storing liquid in the vertical direction; the camera and the laser head for emitting visible light laser are arranged above the liquid level, and the camera is inclined downwards relative to the horizontal plane to face the background; the optical axis of the camera forms a certain included angle with the horizontal plane; then, detecting a moving target of the visible laser spot to obtain a moving track of the visible laser spot; and then acquiring the corner coordinates corresponding to the liquid level position, and solving the actual liquid level value based on the corner coordinates. The method has the disadvantage of being expensive.
Patent No. CN208877510U discloses a monitoring device for venous transfusion, in which a transfusion mechanism is provided with an amplifying structure, the state of liquid in a transfusion tube can be amplified, in addition, the detection mode adopts image acquisition, the liquid level can be clearly detected, in addition, a retaining ring is provided, a hose is buckled, and the shaking is reduced. The method is carefully designed, but also increases the cost of the equipment.
Disclosure of Invention
The invention aims to provide a venous transfusion liquid level detection method based on image processing aiming at the defects of the prior art, so that the change condition of the liquid level during venous transfusion can be known in time, and convenience can be provided for medical care personnel and patients.
The invention provides a venous transfusion liquid level detection method based on image processing, which is characterized by comprising the following steps of:
s1: collecting the picture of the infusion bottle during infusion by a camera;
s2: processing the picture, and carrying out target detection on the infusion bottle through an SSD model;
s3: detecting the liquid level of the detected target infusion bottle in the S2 by adopting a projection method, and detecting the position of the liquid level of the infusion bottle;
s4: and judging whether the liquid level reaches a warning line.
In the above technical solution, the step of establishing the SSD model in step S2 is as follows:
shooting pictures of the infusion bottle with different postures, directions, distances, brightness, definition and angles by using a camera to form a sample set;
making a data set through the sample set; marking an infusion bottle in a picture of a sample set by a marking tool, processing the picture to generate an xml file, and converting the xml file into a TF format file as a data set;
vgg-16.ckpt, setting hyper-parameters, and sending the data set into an SSD model file for training and learning;
after the SSD model is generated after training, the SSD model is sent to a test set for detection, and the result of model training is observed.
In the above technical solution, step S3 specifically includes the following steps: preprocessing the image of the target infusion bottle, including graying, image filtering and pooling; detecting the overall outline of the infusion bottle through Canny edge detection; acquiring a plurality of possible liquid level position information according to the overall outline of the infusion bottle; whether each liquid level position information is a temporary qualified line or not is judged by scanning the accumulated number of the edge points of each liquid level position information in the horizontal direction (the standard of the qualified line is that the accumulated number of the edge points reaches more than one tenth of the whole line); judging all the qualified lines one by one, and judging whether the qualified lines are larger than a preset value or not;
if only one qualified line is larger than the preset value, the liquid level position information represented by the qualified line is judged to be the position of the current liquid level;
if the qualified lines are larger than a preset value, sorting the qualified lines according to length; preferentially judging the longest qualified line; comparing the liquid level position represented by the qualified line with the liquid level of the previous frame of picture, judging that the liquid level position is not the position of the liquid level if no position change exists, recording the position which is not the liquid level and the liquid level length, and then continuously judging the next qualified line;
otherwise, the position of the liquid level is judged and the length of the liquid level is recorded.
In the above technical solution, the step S4 specifically includes the following steps:
calculating the length of a liquid level line of horizontal projection by an integral projection method, and setting the length of an alarm horizontal liquid level; when the length of the liquid level line of the horizontal projection is judged to be smaller than the length of the alarm horizontal liquid level, the fact that the transfusion is about to end is indicated, and a warning prompt is sent out; otherwise, the infusion is not completed.
In the above technical solution, the integral projection method projects an image from two directions, i.e. horizontal and vertical directions, and the horizontal projection h (n) and the vertical projection v (m) are respectively defined as follows:
wherein, (N, M) represents the position of the pixel, I (N, M) represents the gray value of the pixel, N represents the number of pixels in the whole row, and M represents the number of pixels in the whole column.
In the above technical solution, in step S3, the data amount of the internal picture obtained by canny edge detection is reduced by mean pooling, and convolution operation is performed on the picture by using 3 × 3 kernels, so that the content edge in the picture becomes a straight line.
In the above technical solution, in step S2, a target data set is input, feature extraction is performed through a basic convolutional neural network of a VGG16 pre-training model file, prior frames are generated in a feature extraction layer and a partial convolutional neural network layer, the prior frames and a top network are associated, an additional convolutional layer is added to the basic network to predict frame offset and a target category in the prior frames, and a non-maximum value is used to suppress a screening result.
In the above technical solution, in step S2, the hyper-parameters that need to be set for training the SSD model include: category number, tag number, maximum number of steps performed; in the training process of the SSD model, the model is evaluated by entering a TensorBoard visual interface and observing the change of the loss rate and the learning rate: the smaller the loss rate is, the more stable the loss rate is, the model is trained well, and the iteration number also reaches the appropriate requirement. With the increase of the training steps, the average loss rate value is in a descending trend, and the fluctuation amplitude of the average loss rate value is gradually reduced, so that the error between the model prediction sample and the real sample is evaluated; the learning rate is gradually reduced along with the increase of the training steps; the smaller the learning rate is, the more accurate the feature learning is, so as to evaluate the accuracy of model training.
The invention has the beneficial effects that the venous transfusion liquid level detection method based on image processing is provided for solving the problem of low precision of venous transfusion liquid level detection. According to the invention, a deep learning-based SSD (Single Shot Multi Box Detector) algorithm is adopted, the VGG16 is used as a basic network for feature extraction, a multi-scale frame mechanism is adopted for local feature learning, a Non-Maximum Suppression (NMS) is adopted to screen out a default frame, and finally, the position and the class label of a target frame are output, so that the target detection is carried out on the infusion bottle, and the position of the infusion bottle is conveniently and quickly positioned; the liquid level of the infusion bottle is detected by adopting an integral projection method based on image processing, the position of a liquid level line can be accurately and quickly detected, and therefore, the alarm line threshold value is judged according to the projection length of the liquid level line, and whether the alarm state is reached or not is judged. The invention successfully combines the SSD algorithm with the integral projection method, improves the precision and time of the intravenous infusion liquid level detection, and greatly improves the working efficiency of medical care personnel.
Drawings
FIG. 1 is a system block diagram of the present invention
FIG. 2 is a flow chart of the detection of the present invention
FIG. 3 is a flow chart of the SSD algorithm
FIG. 4a is an original image to be detected, and FIG. 4b is a result graph of object detection
FIG. 5 shows the results (parts) of the multiple tests of the embodiment
FIG. 6 is a flow chart of liquid level detection by projection method
FIGS. 7a-7c are graphs showing the effect of the liquid level detection of the present invention (group 1)
FIGS. 8a-8c are graphs of the effectiveness of the level detection of the present invention (group 2).
Detailed Description
The invention will be further described in detail with reference to the following drawings and specific examples, which are not intended to limit the invention, but are for clear understanding.
As shown in fig. 1, which is a block diagram of the intravenous infusion liquid level detection system of the present scheme, the frame is composed of a monitoring object (infusion bottle), a camera, and an image processing terminal:
(1) detecting an object: the real-time infusion liquid level of a single patient intravenous infusion bottle in the infusion chamber;
(2) a camera: the camera collects data of the infusion bottle and transmits the shot picture to the image processing terminal;
(3) an image processing terminal: and the PC end processes the pictures transmitted by the camera, and performs target detection and liquid level detection on the infusion bottle by adopting an SSD algorithm based on deep learning and an integral projection method based on image processing.
As shown in fig. 2, the process chart of the intravenous infusion liquid level detection of the present scheme is divided into the following steps:
(1) image acquisition: and a camera is used for taking a sample set from different angles such as shooting posture, direction, distance, brightness, definition and the like. The infusion bottle data set collection can only be carried out by self shooting because the public data set can not be obtained, and pictures of different liquid levels, angles and distances of the infusion bottle are shot as many as possible. A mobile phone rear camera with the resolution of 800 ten thousand and a back-illuminated/BSI CMOS is supposed to be adopted. Sampling and photographing the infusion state of the infusion bottle, and transmitting the sampled and photographed infusion state to the PC terminal through the network. The infusion bottle is shot and sampled at different illumination (bright or dark) and at different distances of 1 meter and 2 meters and at different angles.
(2) Target detection: and processing the picture, and carrying out target detection on the infusion bottle through an SSD model. In view of the complex environment, the infusion bottle needs to be quickly detected, a large amount of data of the infusion bottle is collected through an SSD target detection algorithm based on deep learning, model training is carried out on the data, and finally target detection is carried out. This step is critical to facilitate the deployment of subsequent jobs. Firstly, a data set is manually made, and a LabLeImage sample labeling tool is used for manually labeling a shot image. And generating an xml file after processing the picture, and then converting the xml file into a TF format file. And downloading a VGG16 model file and setting the hyper-parameters. Sending the training set into an SSD model for training and learning, extracting features of a VGG16 basic network, generating different default frames by a multi-scale frame mechanism, then carrying out intersection-comparison threshold screening, and outputting detection frames, categories and confidence scores through non-maximum value inhibition; after the SSD model is trained, the SSD model is sent to a test set for detection, and the quality of model training is observed.
The artifact labeling uses the label _ image tool to click on the Creat Rectbox, a small box of objects as possible, to learn more carefully during the training process. And (3) proportionally distributing a training verification set by the data set: test set 4: 1, the unified image size is 300 × 300.
The SSD model is established by the following steps:
shooting pictures of the infusion bottle with different postures, directions, distances, brightness, definition and angles by using a camera to form a sample set;
making a data set through the sample set, marking an infusion bottle in a picture of the sample set through a marking tool, processing the picture to generate an xml file, and converting the xml file into a TF format file as the data set;
vgg-16.ckpt, setting hyper-parameters, and sending the data set into an SSD model file for training and learning;
after the SSD model is generated after training, the SSD model is sent to a test set for detection, and the quality of model training is observed.
The training SSD model requires setting parameters: category number, tag number, maximum number of steps performed, etc. Limited to computer configuration, the maximum number of iteration steps set herein: max _ number _ of _ steps 10000, leaving _ rate 0.001, and batch _ size 4.
The hyper-parameters required to be set for training the SSD model comprise: category number, tag number, maximum number of steps performed; in the training process of the SSD model, the parameter curve of the evaluation model can be intuitively seen to continuously change along with the increase of the training steps by means of a TensorBoard visual interface. The parameters of the evaluation model which are important are loss rate and learning rate. The smaller the loss rate is, the more stable the loss rate is, the model is trained well, and the iteration number also reaches the appropriate requirement. With the increase of the number of steps, the average loss rate is in a descending trend, and the fluctuation amplitude is gradually reduced, so that the error between the model prediction sample and the real sample is evaluated. The learning rate gradually decreases as the number of steps increases; the smaller the learning rate is, the more accurate the feature learning is, so as to evaluate the accuracy of model training.
(3) Liquid level detection: and carrying out preprocessing on the image, including graying processing, image filtering and pooling.
Detecting the overall outline of the infusion bottle through Canny edge detection; since the liquid level is always horizontal and longest in the local area of the bottle, the pooled picture is detected by the canny edge, and although there is interference, the general information of the liquid level is still preserved. Whether the pixel points are temporary qualified lines is judged by scanning the accumulated number of the points (namely the detected edge points) with the gray value of 255 of the pixel points in the horizontal direction, and all the qualified lines are judged one by one to see whether the gray values are larger than a preset value, wherein the gray values are set to be 0.3 per srcmagebin.
If only one qualified line is larger than the preset value, the liquid level position information represented by the qualified line is judged to be the position of the current liquid level;
if the qualified lines are larger than a preset value, sorting the qualified lines according to length; preferentially judging the longest qualified line; comparing the liquid level position represented by the qualified line with the liquid level of the previous frame of picture, judging that the liquid level position is not the position of the liquid level if no position change exists, recording the position which is not the liquid level and the liquid level length, and then continuously judging the next qualified line;
otherwise, the position of the liquid level is judged and the length of the liquid level is recorded.
Because of the horizontal edge L of the trademark of the infusion bottleSIs generally positioned at the middle lower part of the infusion bottle and is far larger than the set alarm horizontal liquid level length LTTherefore, useless horizontal edges are removed, and the interference of the horizontal edges of the trademarks on liquid level detection is avoided.
(4) The method adopts an integral projection method to detect the liquid level of the infusion bottle, and the integral projection method mainly projects an image from the horizontal direction and the vertical direction. The horizontal projection and the vertical projection are respectively defined as follows:
wherein, (N, M) represents the position of the pixel, I (N, M) represents the gray value of the pixel, N represents the number of pixels in the whole row, and M represents the number of pixels in the whole column. The horizontal projection is to accumulate the gray values of the pixels in the whole row, the vertical projection is to accumulate the gray values of the pixels in the whole column, and the integral projection function can truly display the total pixel value of the image from the vertical direction or the horizontal direction. The method of horizontal projection is used herein so that the level of the image can be located. The difficulty of liquid level detection is that: the infusion bottle is covered by a trademark, so that the horizontal edge of the trademark can generate an interference item on liquid level detection, and useless horizontal edges are eliminated by reasonably setting the length threshold of the horizontal projection line segment; in addition, the liquid surface is detected directly by canny edge, but the found liquid surface edge has an angle and becomes a 'oblique line', the picture data volume is reduced by pooling of the mean value, and the 'angle' is eliminated by performing convolution operation on the picture by using a 3 × 3 kernel, so that the found edge is 'a straight line'. The method can achieve a better effect to a certain extent and keep key information.
(5) Judging and measuring the alert position: an alarm threshold value is preset, and when the liquid level line approaches the set threshold value, judgment is carried out and an alarm is given out, so that a nurse draws a needle for a patient or changes liquid. And if the threshold value is not reached, continuing to execute the step (1).
As shown in fig. 3, a SSD algorithm flow chart is shown. The main step S2 is: inputting a target data set, carrying out feature extraction through a basic convolutional neural network of a VGG16 pre-training model file, generating a prior frame on a feature extraction layer and a partial convolutional neural network layer, associating the prior frame with a top-level network, adding an additional convolutional layer on the basic network to predict frame offset and a target class in the prior frame, and inhibiting a screening result by using a non-maximum value.
As shown in fig. 4a and 4b, the result is a single-infusion bottle detection result. In this case, box threshold value select _ threshold is 0.4, and non-maximum threshold value nms _ threshold is 0.4. As shown in the figure, fig. 4a is an original image to be detected, and fig. 4b is a result graph of target detection. The category label is output: 1, the accuracy rate is more than 94% and the testing time is within 1.02 s.
As shown in fig. 5, a diagram showing the results of target detection of a plurality of infusion bottles is shown. The average accuracy rate of the 60 test sets is more than 81%, and the average test time is within 1.22 s.
Fig. 6 shows a flow chart of liquid level detection. Preprocessing the picture, including graying, image filtering and pooling; detecting an overall contour through Canny edge detection; and (3) roughly positioning the liquid level by using an integral projection method, detecting the position of the liquid level, judging whether the position reaches a liquid level warning line, and sending an alarm to inform when the liquid level line is smaller than a set threshold value, otherwise, indicating that the transfusion is not finished.
As shown in fig. 7a-c and fig. 8a-c, the effect of the present invention is shown. Fig. 7a and 8a show the results of performing the mean pooling plus the edge detection, fig. 7b and 8b show the results of performing the projection calculation, and fig. 7c and 8c show the results of the final liquid level detection, and it can be seen from the figures that the method can detect the location of the liquid level line well, and in the process, the liquid level detection time is within 0.62s, and the liquid level line can be detected quickly and accurately. Meanwhile, by calculating the length of the liquid level line of the horizontal projection, when the length of the liquid level line of the horizontal projection is smaller than the length of the liquid level at the bottom of the bottle, the completion of the transfusion is indicated, and a warning prompt is sent out; when the length of the liquid level line is larger than the set threshold value, the transfusion is not finished. The difficulty of liquid level detection is as follows: the infusion bottle is covered by a trademark, so that the horizontal edge of the trademark can generate an interference item on liquid level detection, and useless horizontal edges are eliminated by reasonably setting the length threshold of the horizontal projection line segment; in addition, the mean pooling is used, so that the vertical edges can be well removed, the method can achieve a good effect to a certain extent, and key information is reserved.
Through the result display, the SSD algorithm is adopted to detect the target, the complexity is reduced, the real-time performance is improved, the position of the infusion bottle can be detected quickly and accurately, the average test time is within 1.22s, and the average accuracy rate is more than 81%. The method for detecting the liquid level of the infusion solution by adopting the projection method based on image processing is simple to operate, the liquid level line can be accurately and quickly detected, the liquid level detection time is within 0.62s, and the accuracy rate is more than 69%.
Practical researches show that the SSD target detection algorithm based on deep learning and the projection method based on image processing can better realize the liquid level detection function of venous transfusion, and have certain practical value. Can play a beneficial auxiliary role in the medical environment and improve the related working efficiency. In addition, the method can also be applied to other liquid level detection environments such as lake surface water level detection, milk bottle liquid level detection and the like.
Details not described in this specification are within the skill of the art that are well known to those skilled in the art.
Claims (8)
1. A venous transfusion liquid level detection method based on image processing is characterized by comprising the following steps:
s1: collecting the picture of the infusion bottle during infusion by a camera;
s2: processing the picture, and carrying out target detection on the infusion bottle through an SSD model;
s3: detecting the liquid level of the detected target infusion bottle in the S2 by adopting a projection method, and detecting the position of the liquid level of the infusion bottle;
s4: and judging whether the liquid level reaches a warning line.
2. The image processing-based intravenous infusion liquid level detection method according to claim 1, wherein the SSD model establishing step in step S2 is as follows:
shooting pictures of the infusion bottle with different postures, directions, distances, brightness, definition and angles by using a camera to form a sample set;
making a data set through the sample set; marking an infusion bottle in a picture of a sample set by a marking tool, processing the picture to generate an xml file, and converting the xml file into a TF format file as a data set;
vgg-16.ckpt, setting hyper-parameters, and sending the data set into an SSD model file for training and learning;
after the SSD model is generated after training, the SSD model is sent to a test set for detection, and the result of model training is observed.
3. The method for detecting the level of intravenous infusion solution based on image processing as claimed in claim 1, wherein the step S3 comprises the following steps: preprocessing the image of the target infusion bottle, including graying, image filtering and pooling; detecting the overall outline of the infusion bottle through Canny edge detection; acquiring a plurality of possible liquid level position information according to the overall outline of the infusion bottle; judging whether each liquid level position information is a temporary qualified line or not by scanning the accumulated number of edge points of each liquid level position information in the horizontal direction; judging all the qualified lines one by one, and judging whether the qualified lines are larger than a preset value or not;
if only one qualified line is larger than the preset value, the liquid level position information represented by the qualified line is judged to be the position of the current liquid level;
if the qualified lines are larger than a preset value, sorting the qualified lines according to length; preferentially judging the longest qualified line; comparing the liquid level position represented by the qualified line with the liquid level of the previous frame of picture, judging that the liquid level position is not the position of the liquid level if no position change exists, recording the position which is not the liquid level and the liquid level length, and then continuously judging the next qualified line;
otherwise, the position of the liquid level is judged and the length of the liquid level is recorded.
4. The method for detecting the level of intravenous fluid infusion based on image processing as claimed in claim 1, wherein said step S4 specifically comprises the steps of:
calculating the length of a liquid level line of horizontal projection by an integral projection method, and setting the length of an alarm horizontal liquid level; when the length of the liquid level line of the horizontal projection is judged to be smaller than the length of the alarm horizontal liquid level, the fact that the transfusion is about to end is indicated, and a warning prompt is sent out; otherwise, the infusion is not completed.
5. The method of claim 3, wherein the integral projection method projects the image from both horizontal and vertical directions, and the horizontal projection H (n) and the vertical projection V (m) are respectively defined as follows:
wherein, (N, M) represents the position of the pixel, I (N, M) represents the gray value of the pixel, N represents the number of pixels in the whole row, and M represents the number of pixels in the whole column.
6. The method according to claim 1, wherein in step S3, the average pooling is used to reduce the amount of internal picture data obtained by canny edge detection, and a 3 x 3 kernel is used to perform convolution operation with the picture, so that the content edge in the picture becomes a straight line.
7. The method of claim 2, wherein in step S2, the target data set is input, feature extraction is performed through a basic convolutional neural network of a VGG16 pre-training model file, prior frames are generated at a feature extraction layer and a partial convolutional neural network layer, the prior frames and a top-level network are associated, an additional convolutional layer is added to the basic network to predict frame offset and target classes in the prior frames, and a non-maximum value is used to suppress the screening result.
8. The image processing-based intravenous infusion liquid level detection method according to claim 2, wherein in step S2, the hyper-parameters required to be set for training the SSD model comprise: category number, tag number, maximum number of steps performed; in the training process of the SSD model, the model is evaluated by entering a TensorBoard visual interface and observing the change of the loss rate and the learning rate: the smaller the loss rate is, the more stable the loss rate is, the model is trained well, and the iteration times also reach the appropriate requirements; with the increase of the training steps, the average loss rate value is in a descending trend, and the fluctuation amplitude of the average loss rate value is gradually reduced, so that the error between the model prediction sample and the real sample is evaluated; the learning rate is gradually reduced along with the increase of the training steps; the smaller the learning rate is, the more accurate the feature learning is, so as to evaluate the accuracy of model training.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110535142.8A CN113298089A (en) | 2021-05-17 | 2021-05-17 | Venous transfusion liquid level detection method based on image processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110535142.8A CN113298089A (en) | 2021-05-17 | 2021-05-17 | Venous transfusion liquid level detection method based on image processing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113298089A true CN113298089A (en) | 2021-08-24 |
Family
ID=77322461
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110535142.8A Pending CN113298089A (en) | 2021-05-17 | 2021-05-17 | Venous transfusion liquid level detection method based on image processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113298089A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114565848A (en) * | 2022-02-25 | 2022-05-31 | 佛山读图科技有限公司 | Liquid medicine level detection method and system in complex scene |
| CN115457744A (en) * | 2022-09-14 | 2022-12-09 | 广州远正智能科技股份有限公司 | Monitoring and alarming system and method for aluminum liquid leakage of aluminum processing flow distribution disc |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105181082A (en) * | 2015-04-30 | 2015-12-23 | 湖南大学 | Liquid level detection method and liquid level detection device based on visible laser and image processing |
| CN109886359A (en) * | 2019-03-25 | 2019-06-14 | 西安电子科技大学 | Small target detecting method and detection model based on convolutional neural networks |
-
2021
- 2021-05-17 CN CN202110535142.8A patent/CN113298089A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105181082A (en) * | 2015-04-30 | 2015-12-23 | 湖南大学 | Liquid level detection method and liquid level detection device based on visible laser and image processing |
| CN109886359A (en) * | 2019-03-25 | 2019-06-14 | 西安电子科技大学 | Small target detecting method and detection model based on convolutional neural networks |
Non-Patent Citations (1)
| Title |
|---|
| 巴桂: "基于图像处理的静脉输液无液检测", 《中国优秀博硕士学位论文全文数据库(硕士)医药卫生科技辑》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114565848A (en) * | 2022-02-25 | 2022-05-31 | 佛山读图科技有限公司 | Liquid medicine level detection method and system in complex scene |
| CN114565848B (en) * | 2022-02-25 | 2022-12-02 | 佛山读图科技有限公司 | Liquid medicine level detection method and system in complex scene |
| CN115457744A (en) * | 2022-09-14 | 2022-12-09 | 广州远正智能科技股份有限公司 | Monitoring and alarming system and method for aluminum liquid leakage of aluminum processing flow distribution disc |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11145130B2 (en) | Method for automatically capturing data from non-networked production equipment | |
| Wu et al. | Detection of the respiratory rate of standing cows by combining the Deeplab V3+ semantic segmentation model with the phase-based video magnification algorithm | |
| US11751848B2 (en) | Methods and apparatuses for ultrasound data collection | |
| CN113298089A (en) | Venous transfusion liquid level detection method based on image processing | |
| CN108509908B (en) | Pupil diameter real-time measurement method based on binocular stereo vision | |
| CN110766656B (en) | Method, device, equipment and storage medium for screening fundus macular region abnormality | |
| CN112750531A (en) | Automatic inspection system, method, equipment and medium for traditional Chinese medicine | |
| CN113688817A (en) | Instrument identification method and system for automatic inspection | |
| US20210100530A1 (en) | Methods and systems for diagnosing tendon damage via ultrasound imaging | |
| CN111685740A (en) | Heart function parameter detection method and device | |
| US20240029372A1 (en) | Method for automatically capturing data from non-networked production equipment | |
| CN111144506B (en) | Liver bag worm identification method based on ultrasonic image, storage medium and ultrasonic equipment | |
| US11430236B1 (en) | Computer-implemented segmented numeral character recognition and reader | |
| WO2023070946A1 (en) | Measurement device and method based on ultrasound image, and medium and electronic device | |
| WO2022127318A1 (en) | Scanning positioning method and apparatus, storage medium and electronic device | |
| Suksawatchon et al. | Shape Recognition Using Unconstrained Pill Images Based on Deep Convolution Network | |
| Charles et al. | Real-time screen reading: reducing domain shift for one-shot learning. | |
| CN106204609B (en) | The processing of Laser scanning confocal microscope lung image and analysis system and method | |
| FR3110314A1 (en) | Non-intrusive digital monitoring of existing machinery and equipment using machine learning and computer vision. | |
| CN112001896A (en) | Thyroid gland border irregularity detection device | |
| US20230401814A1 (en) | A bottle analysis system | |
| CN111833308B (en) | Respiratory motion monitoring method and monitoring system based on Kinect | |
| CN210990247U (en) | Blood pressure detection system | |
| Ullah et al. | Quantifying consistency of crop establishment using a lightweight U-Net deep learning architecture and image processing techniques | |
| WO2023002503A1 (en) | A system and a method for synthesization and classification of a micro-motion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210824 |
|
| RJ01 | Rejection of invention patent application after publication |