CN113222988A - Infusion tube liquid level identification method based on deep learning and image overdivision - Google Patents

Infusion tube liquid level identification method based on deep learning and image overdivision Download PDF

Info

Publication number
CN113222988A
CN113222988A CN202110633471.6A CN202110633471A CN113222988A CN 113222988 A CN113222988 A CN 113222988A CN 202110633471 A CN202110633471 A CN 202110633471A CN 113222988 A CN113222988 A CN 113222988A
Authority
CN
China
Prior art keywords
image
liquid level
infusion tube
deep learning
method based
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110633471.6A
Other languages
Chinese (zh)
Inventor
林峰
郑子睿
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202110633471.6A priority Critical patent/CN113222988A/en
Publication of CN113222988A publication Critical patent/CN113222988A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20112Image segmentation details
    • G06T2207/20132Image cropping
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30021Catheter; Guide wire

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses an infusion tube liquid level identification method based on deep learning and image overdivision, which belongs to the technical field of intelligent image processing, and is used for judging whether the liquid level of each infusion tube is lower than a critical value or not by monitoring the liquid level conditions of one or more infusion tubes and each infusion tube in a video in real time so as to provide an alarm signal and ensure the infusion safety of patients. The method can identify a plurality of infusion tubes simultaneously, the real-time performance meets the actual requirement, the related medical accidents can be effectively reduced or even eliminated, and the method has high application value.

Description

Infusion tube liquid level identification method based on deep learning and image overdivision
Technical Field
The invention relates to the technical field of intelligent image processing, in particular to a transfusion tube liquid level identification method based on deep learning and image overdivision.
Background
Intravenous infusion is a common medical procedure. It is a medicine supply mode for inputting large dose of injection into body by utilizing the principle of atmospheric pressure and hydrostatic pressure. In the actual medical environment, the problem that the liquid level height of the infusion tube cannot be checked in time due to the fact that patients are numerous, the infusion site is large and the like always troubles nursing staff and patients. Due to the limitation of manpower and material resources, limited intelligent detection equipment and medical care personnel need to be preferentially supplied to critically ill patients for use; the infusion hall of the mild patients can only be managed by a plurality of medical care personnel. In most cases, the patient only can pay attention to the amount of the residual liquid medicine in the infusion tube. Therefore, patients often sleep due to fatigue, and the disease is not discovered until 1 or 2 hours after the liquid medicine is dripped. In order to avoid the occurrence of similar events and to enable patients to have better medical experience, an inexpensive system capable of tracking and detecting the liquid level condition of the infusion tube of each patient in real time is needed.
At present, there are many automatic monitoring systems, for example, chinese patent publication No. CN112604074A discloses a multifunctional automatic transfusion device, which can automatically perform the actions of inserting needle, pulling needle and replacing transfusion bottle. Meanwhile, the automatic infusion adjusting assembly can monitor the infusion speed in real time and control the infusion speed to be always in the range required by the medical advice. Chinese patent publication No. CN112121258A discloses an infusion monitoring device and an infusion monitoring method, which are used when intravenous infusion is performed, and monitor the liquid level of an infusion bottle, the flow rate of a liquid medicine, and whether the liquid medicine drips out of a drip cup by detecting a camera. The invention discloses an intelligent monitoring system and method applied to clinical infusion nursing, which integrates remote alarm, automatic clamping, infusion medicine and patient information display, greatly reduces unnecessary manual infusion detection burden of patients and accompanying personnel, and improves the fluid replacement efficiency of medical personnel. However, these patents require special devices or special sensors, which greatly limit their applications.
The development of computer image and video technology at present provides possibility for automatically tracking the transfusion condition, and the liquid level condition of each transfusion tube is detected in real time by adopting a video image processing and deep learning method. The detection system shoots and records the infusion hall through the camera and uploads the shot and recorded infusion hall to the PC end, and the target object can be screened out by the video processing program of the PC end: the infusion tubes detect the liquid level of each infusion tube in real time, and send out alarm signals when the liquid level of the infusion tubes is lower than a critical value so as to ensure the infusion safety of patients.
Disclosure of Invention
The invention aims to provide an infusion tube liquid level identification method based on deep learning and image overdivision, which judges whether the liquid level of each infusion tube is lower than a critical value or not by monitoring the liquid level conditions of one or more infusion tubes and each infusion tube in a video in real time so as to provide an alarm signal and ensure the infusion safety of patients. In order to achieve the above object, the infusion tube liquid level identification method based on depth learning and image super-resolution provided by the invention specifically comprises the following steps as shown in fig. 1:
1) intercepting image data from a camera;
2) carrying out infusion tube detection by using YOLOv 5;
3) judging whether the infusion tube is detected or not, and if not, going to step 9);
4) cutting the image according to the boundary information and reserving the original image;
5) amplifying the cut image by using a DRN-master;
6) carrying out liquid level detection on the amplified image by using YOLOv 5;
7) judging whether the liquid level is detected or not, and if not, going to step 9);
8) labeling on the original image according to the information of the bounding box, as shown in fig. 5;
9) and outputting the image in real time.
According to the technical scheme, high-definition shooting and recording equipment is used for recording, the angle of the camera can be rotated or fixed, after a relatively stable video image is obtained, the video information is preprocessed, then the trained model is used for classification and judgment, and alarm information is given according to the judgment result. The method can identify a plurality of infusion tubes simultaneously, the real-time performance meets the actual requirement, the related medical accidents can be effectively reduced or even eliminated, and the method has high application value.
Preferably, in step 1), image data is intercepted from a camera; the sample set preparation method specifically comprises two steps:
step 1-1) intercepting image frames in a real-time video; the video recording was performed when 50%, 35%, 20% and less than 5% of the liquid remained in the infusion tube, and the video recording frame rate was 30fps and the pixel was 720p, respectively. During recording, the factors such as shooting angle, distance and the like are changed, and then the picture is intercepted from the video through software free studio. An example of a sample of a video set is shown in fig. 2.
Step 1-2) sample labeling; and manually labeling the shot image by using a labelImg sample labeling tool, and programming to convert the xml file into a txt file required by YOLO.
Preferably, in step 2), the infusion tube is detected by using YOLOv 5; in practical application, one or more cameras are often adopted to monitor the condition of the whole infusion space, the definition of some monitoring devices is often low, the difficulty of detecting the infusion tube from a video is high, the YOLOv5m.pt of YOLOv5 is adopted as the weight of a model, the model gives consideration to the size of a weight file, the difficulty of training, the detection accuracy and the detection speed, and the performance is balanced.
Preferably, in step 4), the image is cut according to the boundary information, and the original image is reserved; and cutting the detected infusion tube image according to the boundary frame for subsequent image processing, liquid level detection and the like.
Preferably, in step 5), the cropped image is enlarged by a DRN-master; in order to improve the identification capability of the system on small targets such as liquid level and the like, the DRN-master image super-resolution algorithm is adopted in the embodiment, the image of the infusion tube cut in the step 4) is amplified, and characteristic information such as boundary, liquid level and the like is highlighted.
The DRN-master designs a network structure of dual regression, which not only provides additional training supervision, but also allows the algorithm to be trained by using non-one-to-one corresponding high and low resolution images. The network structure of the DRN-master is shown in fig. 3, and the effect of increasing the picture definition by 4 times using the DRN-master is shown in fig. 4 in comparison.
Preferably, in step 6), the amplified image is subjected to liquid level detection by using YOLOv 5; the level detection in this example uses YOLOv5s.pt from YOLOv5 as model weight. The depth of the network of yolov5s and the width of the characteristic diagram are small, so the training difficulty is low, the detection speed is high, and the accuracy can meet the requirement.
Compared with the prior art, the invention has the beneficial effects that:
the invention relates to an infusion tube liquid level identification method based on deep learning and image overdivision, which judges whether the liquid level of each infusion tube is lower than a critical value or not by monitoring the liquid level conditions of one or more infusion tubes and each infusion tube in a video in real time so as to provide an alarm signal and ensure the infusion safety of patients. The method can identify a plurality of infusion tubes simultaneously, the real-time performance meets the actual requirement, the related medical accidents can be effectively reduced or even eliminated, and the method has high application value.
Drawings
FIG. 1 is a flow chart of an infusion tube liquid level identification method based on depth learning and image hyper-resolution in an embodiment of the present invention;
FIG. 2 is an example of a sample video set according to an embodiment of the present invention;
FIG. 3 is a network structure of a DRN-master in an embodiment of the present invention;
FIG. 4 is a comparison graph of the DRN-master super resolution processing effect (magnified 4 times, right is the input image) in the embodiment of the present invention, wherein (a) is the input image and (b) is the processed image;
FIG. 5 is an image labeled according to bounding box information in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described with reference to the following embodiments and accompanying drawings.
Examples
Referring to fig. 1 to 5, a method for identifying a liquid level of an infusion tube based on depth learning and image hyper-resolution according to the present embodiment is shown in fig. 1, and includes the following steps:
and step S100, intercepting image data from a camera. And judging whether the sample is made into a sample set or not, and if not, going to the step S200.
Step S110, intercepting image frames in a real-time video; the video recording was performed when 50%, 35%, 20% and less than 5% of the liquid remained in the infusion tube, and the video recording frame rate was 30fps and the pixel was 720p, respectively. During recording, the factors such as shooting angle, distance and the like are changed, and then the picture is intercepted from the video through software free studio. An example of a sample of a video set is shown in fig. 2.
Step S120, marking a sample; manually labeling the shot image by using a labelImg sample labeling tool, programming to convert the xml file into a txt file required by YOLO, and performing the following steps according to the sequence of 4: a ratio of 1 generates a training set and a test set. Format of each line within txt:
<object-class><x_center><y_center><width><height>。
in step S200, the infusion tube is detected by YOLOv 5. In practical application, one or more cameras are often adopted to monitor the condition of the whole infusion space, the definition of some monitoring devices is often low, the difficulty in detecting the infusion tube directly from a video is high, YOLOv5m.pt of YOLOv5 is adopted as the model weight, the model gives consideration to the size of a weight file, the difficulty in training, the detection accuracy and the detection speed, and the performance is balanced. In this example, under yolov5s.pt weight, the maximum set batch size (batch-size) is 21, the input picture size is 352 × 352, and the total training step number is 800.
Step S300, judging whether the infusion tube is detected, if not, going to step S900.
And step S400, cutting the image according to the boundary information and reserving the original image.
And step S500, amplifying the cut image by using a DRN-master. In order to improve the identification capability of the system on small targets such as liquid level and the like, the DRN-master image super-resolution algorithm is adopted in the embodiment, the image of the infusion tube cut in the step 4) is amplified, and characteristic information such as boundary, liquid level and the like is highlighted.
The DRN-master designs a network structure of dual regression, which not only provides additional training supervision, but also allows the algorithm to be trained by using non-one-to-one corresponding high and low resolution images. The network structure of the DRN-master is shown in fig. 3, and the effect of increasing the picture definition by 4 times using the DRN-master is shown in fig. 4 in comparison.
In step S600, the level of the enlarged image is detected by YOLOv 5. The level detection in this example uses YOLOv5s.pt from YOLOv5 as model weight. The depth of the network of yolov5s and the width of the characteristic diagram are small, so the training difficulty is low, the detection speed is high, and the accuracy can meet the requirement.
Step S700, judging whether the liquid level is detected, if not, going to step S900.
In step S800, the original image is marked according to the bounding box information, as shown in fig. 5.
And step S900, outputting the image in real time.

Claims (6)

1. An infusion tube liquid level identification method based on deep learning and image overdivision is characterized by comprising the following steps:
1) intercepting image data from a camera;
2) carrying out infusion tube detection by using YOLOv 5;
3) judging whether the infusion tube is detected or not, and if not, going to step 9);
4) cutting the image according to the boundary information and reserving the original image;
5) amplifying the cut image by using a DRN-master;
6) carrying out liquid level detection on the amplified image by using YOLOv 5;
7) judging whether the liquid level is detected or not, and if not, going to step 9);
8) marking on the original image according to the information of the boundary frame;
9) and outputting the image in real time.
2. The infusion tube liquid level identification method based on the deep learning and the image super-resolution as claimed in claim 1, wherein in the step 1), image data is intercepted from a camera; the sample set preparation method specifically comprises two steps:
step 1-1) intercepting image frames in a real-time video; during recording, the factors such as shooting angle, distance and the like are changed, and then the picture is intercepted from the video through software free studio.
Step 1-2) sample labeling; and manually labeling the shot image by using a labelImg sample labeling tool, and programming to convert the xml file into a txt file required by YOLO.
3. The infusion tube liquid level identification method based on the deep learning and the image super-resolution as claimed in claim 1, wherein in the step 2), the infusion tube is detected by YOLOv 5; this example uses YOLOv5m.pt from YOLOv5 as the model weight.
4. The infusion tube liquid level identification method based on the deep learning and the image super-resolution as claimed in claim 1, wherein in the step 4), the image is cut according to the boundary information, and the original image is reserved; and cutting the detected infusion tube image according to the boundary frame for subsequent image processing, liquid level detection and the like.
5. The infusion tube liquid level recognition method based on the deep learning and the image super-resolution as claimed in claim 1, wherein in the step 5), the cut image is amplified by a DRN-master; in order to improve the identification capability of the system on small targets such as liquid level and the like, the DRN-master image super-resolution algorithm is adopted in the embodiment, the image of the infusion tube cut in the step 4) is amplified, and characteristic information such as boundary, liquid level and the like is highlighted.
6. The infusion tube liquid level identification method based on the deep learning and the image super-resolution as claimed in claim 1, wherein in the step 6), the amplified image is subjected to liquid level detection by using YOLOv 5; the level detection in this example uses YOLOv5s.pt from YOLOv5 as model weight.
CN202110633471.6A 2021-06-07 2021-06-07 Infusion tube liquid level identification method based on deep learning and image overdivision Pending CN113222988A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110633471.6A CN113222988A (en) 2021-06-07 2021-06-07 Infusion tube liquid level identification method based on deep learning and image overdivision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110633471.6A CN113222988A (en) 2021-06-07 2021-06-07 Infusion tube liquid level identification method based on deep learning and image overdivision

Publications (1)

Publication Number Publication Date
CN113222988A true CN113222988A (en) 2021-08-06

Family

ID=77083071

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110633471.6A Pending CN113222988A (en) 2021-06-07 2021-06-07 Infusion tube liquid level identification method based on deep learning and image overdivision

Country Status (1)

Country Link
CN (1) CN113222988A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115019456A (en) * 2022-05-31 2022-09-06 浙江大学 Ward calling system and calling method thereof
CN117392135A (en) * 2023-12-12 2024-01-12 深圳市普朗医疗科技发展有限公司 Injection dose analysis method and system based on image

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109635666A (en) * 2018-11-16 2019-04-16 南京航空航天大学 A kind of image object rapid detection method based on deep learning
CN110188807A (en) * 2019-05-21 2019-08-30 重庆大学 Tunnel pedestrian target detection method based on cascade super-resolution network and improvement Faster R-CNN

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109635666A (en) * 2018-11-16 2019-04-16 南京航空航天大学 A kind of image object rapid detection method based on deep learning
CN110188807A (en) * 2019-05-21 2019-08-30 重庆大学 Tunnel pedestrian target detection method based on cascade super-resolution network and improvement Faster R-CNN

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
YONG GUO 等: "Closed-Loop Matters: Dual Regression Networks for Single Image Super-Resolution", 《2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 *
ZEYONG HUANG 等: "Infusion port level detection for intravenous infusion based on Yolo v3 neural network", 《MATHEMATICAL BIOSCIENCES AND ENGINEERING》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115019456A (en) * 2022-05-31 2022-09-06 浙江大学 Ward calling system and calling method thereof
CN117392135A (en) * 2023-12-12 2024-01-12 深圳市普朗医疗科技发展有限公司 Injection dose analysis method and system based on image
CN117392135B (en) * 2023-12-12 2024-03-01 深圳市普朗医疗科技发展有限公司 Injection dose analysis method and system based on image

Similar Documents

Publication Publication Date Title
CN113222988A (en) Infusion tube liquid level identification method based on deep learning and image overdivision
US11145130B2 (en) Method for automatically capturing data from non-networked production equipment
CN108830824A (en) Transfusion liquid facial vision detects alarm system and its detection method
WO2011094639A3 (en) Methods and systems for analyzing, prioritizing, visualizing, and reporting medical images
US7853063B2 (en) Method for producing a cropped medical image
KR20090006295A (en) Apparatus, system and method for examination of cell slide and readable medium thereof
CN113891072B (en) Video monitoring and anomaly analysis system and method based on hundred million-level pixel data
US20070140538A1 (en) Method for processing unenhanced medical images
JP2017069963A (en) Data collection system of display panel and operation panel, device, method, program and recording medium
CN101546376A (en) Human body biological information acquisition system, human face photo acquisition and quality inspection system and method
US11900549B2 (en) Method for automatically capturing data from non-networked production equipment
CN113610077A (en) System method and equipment for monitoring and analyzing dissolution behavior by using artificial intelligence image recognition technology
CN113485555B (en) Medical image film reading method, electronic equipment and storage medium
CN109223178B (en) Cavity mirror intelligent edge computing system with target positioning function
CN113537064A (en) Weak pig automatic detection marking method and system
CN100384222C (en) Method and device for adjusting camera position
KR102423255B1 (en) System and method for integrated management synchronizing sensor information and video information based on the internet of things
CN112862752A (en) Image processing display method, system electronic equipment and storage medium
CN116912900A (en) Method and device for detecting falling of breathing mask of patient
CN115514918B (en) Remote video method, cloud platform, communication mobile platform and storage medium
CN110599471A (en) Rain gauge horizontal monitoring system based on image processing and detection method thereof
US20230011929A1 (en) Interface between heart pump controller database and hospital
CN117524448A (en) Medical image display mode adjusting method and device
CN105286897A (en) Medical image diagnosis system
CN113901934A (en) Intelligent visual detection method, system and device for large infusion package product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20210806