CN112932482A

CN112932482A - Puncture technology based on monocular camera recognition

Info

Publication number: CN112932482A
Application number: CN202110119798.1A
Authority: CN
Inventors: 陈晨
Original assignee: Nantong Pakion Medical Material Co ltd
Current assignee: Nantong Pakion Medical Material Co ltd
Priority date: 2021-01-28
Filing date: 2021-01-28
Publication date: 2021-06-11

Abstract

A puncture technology based on monocular camera identification mainly adopts a depth learning model to assist in generating a depth image, the model mainly comprises a generation model and a judgment model, the generation model mainly has the main function of generating another image according to a shot image, a judgment network mainly judges the generated image, the quality of the generated image is judged, an image which is highly matched with a binocular distance measurement principle is generated in an antagonistic mode, and the pose of a puncture needle is obtained in real time. The method greatly reduces the cost of equipment, does not need an expensive high-precision coding motor for assistance, only needs one camera, has close precision, can realize millimeter-scale positioning, is beneficial to improving the intellectualization and automation of hospitals, reduces the contact of medical staff and patients, effectively inhibits the transmission of infectious diseases, greatly reduces the manual operation and saves the human resources.

Description

Puncture technology based on monocular camera recognition

Technical Field

The invention relates to an autonomous puncture device under a medical background, which designs and realizes a puncture technology, can autonomously complete vein searching and puncture, and belongs to the field of medical automation and medical image processing.

Background

At present, in the field of vein identification and puncture, most devices are specially used for vein enhancement and imaging, wherein vein ultrasonic imaging is a mainstream vein imaging technology, but the technology is poor in imaging quality and poor in stability, and still needs to be assisted by experts for distinguishing to find the position of a vein. Venous access difficulties can have adverse effects on the patient, and multiple punctures increase anxiety and pain for the patient. Clinical studies have shown that 51% of children and 83% of children suffer from intense pain when receiving conventional venipuncture. Statistically, when a child in a hospital sees a puncture needle, pain and panic may occur, 74% of which are caused by difficulty in accessing veins. At the same time, the difficulty of venous access can lead to serious complications of multiple punctures, which, while increasing the trauma of the patient, may carry risks of phlebitis, venous infections, etc., which will directly increase the additional costs. In addition, the rescue of patients is delayed due to the difficulty in vein access, the vein puncture difficulty of children with the difficulty in vein access is extremely high, and the average time for completing the puncture is 23 min. During this time, the drugs cannot enter the veins, and the veins also affect the throughput of the drugs, thereby delaying the optimal treatment time and delaying rescue.

In the traditional intravenous injection process, in order to improve the success rate of venipuncture, doctors can adopt a method of beating the back of the hand and tightening the arms so as to facilitate the observation of the vein prominence of a patient. However, the slight pain of beating and the tight pulse pressing belt bring certain trouble to patients while increasing the injection procedures of doctors. In addition, due to the large difference of the vein distribution among patients, the method shows great limitation when patients with difficult vein access are encountered. Meanwhile, some nurses may make mistakes in the intravenous injection process to cause trouble for patients, in order to solve the problems, the patent discloses a vision-based needle pose estimation method, and the deep learning is widely applied to various scenes nowadays, so that the patent utilizes a single camera to obtain vein images and combines a neural network to generate depth information, thereby obtaining the pose information of the needle, not only can realize high-precision venipuncture, reduce medical disputes, but also does not need a higher-price encoder, thereby saving the cost, and further widening the benefit range of the patent.

In the special period of epidemic outbreak, this patent can replace medical personnel to carry out venipuncture, has alleviated personnel's pressure on the one hand, and on the other hand can alleviate medical personnel's risk of infecting, alleviates the vein and seeks the misery of the difficult patient, has also avoided because of medical personnel's mood fluctuation, the puncture error that causes such as tired. Under the normal state, the puncture needle is also suitable for the scenes that puncture is carried out on people with difficulty in finding veins, such as obese people, infants and the like, at present, the medical treatment gradually trends to intelligent automation development, and the puncture needle is expected to be used as a representative form of medical automation in the future.

Disclosure of Invention

This patent has mainly realized a syringe needle position appearance estimation and puncture method based on vision, and need not pass through the puncture of expensive encoder in order to realize the millimeter level, to traditional scheme, in order to reduce the complexity of system, the camera of common way is mostly to gather image information, give the three-dimensional support, accomplish puncture work by support control syringe needle, this often needs expensive encoder in order to realize accurate puncture effect, for this reason, this patent has proposed a scheme based on neural network's monocular camera generates the depth map and realizes accurate puncture.

The depth estimation network mainly comprises a generation model and a discrimination model, wherein the generation model is mainly used for generating another image according to a shot image, the generated image and the shot image are equivalent to images collected by a left camera and a right camera of a binocular camera, and a depth image is generated through the two images; the judgment network mainly judges the generated images, judges the quality of the generated images, and generates images highly conforming to the binocular range finding principle in an antagonistic mode so as to acquire the pose of the puncture needle in real time.

When puncture operation is carried out, the diameter of a blood vessel is assumed to be 2R, the length of a metal part of a puncture needle is assumed to be L, according to experience, the length of the puncture needle immersed into the blood vessel is between L/2 and 2/3L, the puncture angle theta is between 15 and 30 degrees, and in order to improve the success rate of puncture, the following puncture strategies are designed:

1. searching the thickest part of the blood vessel on the back of the hand by an infrared camera, calibrating the blood vessel and determining a puncture point;

2. judging the depth, namely the distance, of the puncture point in the equipment through a camera and a depth estimation network, and controlling the puncture needle to reach a specified position to prepare for puncture;

3. after the puncture needle reaches the designated position, the puncture motor adjusts the puncture angle formed by the puncture needle and the back of the hand, and after the angle preparation is finished, the puncture is started;

4. firstly, pre-puncturing according to the diameter 2R of a blood vessel, wherein the pre-puncturing depth is 1/3 of the diameter of the blood vessel, the length of a puncture needle to be punctured is 2R/3sin theta, and the purpose of puncturing is to ensure that the puncture needle can be accurately inserted into the blood vessel and cannot be inserted too deeply;

and then, continuously puncturing, different from pre-puncturing, ensuring that the puncture needle cannot penetrate through a blood vessel in the subsequent puncturing step, setting the depth of a puncturing target to be R for the purpose, and simultaneously ensuring that the length of the puncture needle immersed into the skin is L/2-2/3L, so that an angle alpha epsilon [0 degrees ], 15 degrees ] and a depth D epsilon [2R/3, R ] and the length L of the puncture needle immersed into the skin epsilon [ L/2,2L/3] model are established:

alpha as arcsin D/l (formula I)

When the puncture reaches the designated depth, the puncture is finished.

The puncturing method based on the neural network model greatly reduces the complexity of the equipment and simultaneously reduces the investment cost of the equipment.

Drawings

FIG. 1 is a drawing of an abstract of the specification

FIG. 2 is a depth image generation map

FIG. 3 is a puncture flowchart

Detailed Description

The technical scheme of the patent is further explained in detail with reference to the attached drawings as follows:

firstly, a patient takes a transfusion bottle of the patient from a medicine taking window to come before the patient comes to equipment, the transfusion bottle is hung on the equipment, a transfusion pipe is inserted, then a hand to be injected is placed at a specified position in the equipment, a camera collects an image of the equipment for the first time and inputs the image into a trained generation network, the generation network generates another pair of images according to the shot image, the generated images are input into a discrimination network, the quality of the generated images is judged, basically, the puncture requirement can be met for the trained model, images which are highly matched with a binocular distance measurement principle are generated in an antagonistic mode, the generated images and the shot images are equivalent to images collected by a left camera and a right camera of a binocular camera, and the pose and the hand state of a puncture needle are obtained in real time by generating depth images through the two images.

After the acquisition is finished, acquiring a blood vessel image of the back of the hand by using infrared equipment, marking a thicker part in the image as a puncture point to prepare for subsequent puncture, controlling a puncture needle to reach a specified position by using a three-dimensional support after the puncture is determined, preparing for puncture, wherein the diameter of a blood vessel is 2R, the length of a metal part of the puncture needle is L, the length of the puncture needle immersed in the blood vessel is L/2-2/3L, and the puncture angle theta is determined, searching for the thickest part on the blood vessel of the back of the hand by using an infrared camera, calibrating and determining the puncture point; judging the depth, namely the distance, of the puncture point in the equipment through a camera and a depth estimation network, and controlling the puncture needle to reach a specified position to prepare for puncture; after the puncture needle reaches the designated position, the puncture motor adjusts the puncture angle formed by the puncture needle and the back of the hand, and after the angle preparation is finished, the puncture is started; first, according to the diameter 2R of the blood vessel, pre-puncture is carried out, the pre-puncture depth is 1/3 of the diameter of the blood vessel, and the puncture needle is inserted into the blood vessel at the time

The purpose of puncturing is to ensure that the puncture needle can not be inserted too deeply while being accurately inserted into the blood vessel; then, the puncture is continued, different from the pre-puncture, the follow-up puncture step needs to ensure that the puncture needle cannot penetrate through the blood vessel, for this reason, the puncture target depth is set to be R, and meanwhile, the length of the puncture needle immersed into the skin is ensured to be L/2-2/3L, so that the angle is established

And depth

And length of the needle submerged in the skin

Model: when the puncture reaches the designated depth, the puncture is finished.

Although the embodiments have been described, once the basic inventive concept is obtained, other variations and modifications of these embodiments can be made by those skilled in the art, so that the above embodiments are only examples of the present invention, and not intended to limit the scope of the present invention, and all equivalent structures or equivalent processes using the contents of the present specification and drawings, or any other related technical fields, which are directly or indirectly applied thereto, are included in the scope of the present invention.

Claims

1. A puncture technology based on monocular camera recognition is characterized in that a deep learning technology is adopted to generate a depth image, the quality of the generated image is improved through a network countermeasure mode, the generated image is combined with an image acquired by a camera to generate the depth image for judging the pose of a puncture needle, and then puncture is accurately performed according to a certain puncture technology.

2. A puncture technique based on monocular camera identification as set forth in claim 1, wherein the camera used is a near-infrared camera, and in order to eliminate interference, a high-power near-infrared lamp is used as a light source in order to enhance the image effect obtained by utilizing the characteristic that hemoglobin in the vein absorbs near-infrared light strongly.

3. The monocular camera recognition-based puncturing technique of claim 1, wherein the countermeasure network is for generating an image having a certain rule, and the image and the original image constitute a left and right image of a binocular camera.

4. The monocular camera recognition-based penetration technique of claim 1, wherein the depth image is generated based on a method of generating a depth image by a binocular camera.

5. The monocular camera recognition-based penetration technique of claim 1, wherein the generated depth image is used to estimate the pose and position of the penetration needle while measuring the height of the back of the hand.

6. The method for realizing the puncture technology based on monocular camera recognition according to claim 1, comprising the following steps:

firstly, acquiring an image by equipment, and inputting the image into a generation network to generate the image;

inputting the generated image into a judging network for quality judgment, if the generated image does not accord with the standard, regenerating the generated image, and if the generated image accords with the standard, combining the generated image with an original image collected by a camera to generate a depth image;

judging the posture and position of the puncture needle and the height of the back of the hand through the depth image, and simultaneously acquiring the thickest point of the vein through the infrared image and marking the point as the puncture point;

and step four, controlling the puncture needle to reach the designated position by the three-dimensional support, performing pre-puncture at first, then slowly puncturing according to a certain angle until the puncture needle reaches the preset position, and finishing puncture.