CN111481777A - Non-contact vein imaging virtual reality glasses based on memristor calculation - Google Patents
Non-contact vein imaging virtual reality glasses based on memristor calculation Download PDFInfo
- Publication number
- CN111481777A CN111481777A CN202010371109.1A CN202010371109A CN111481777A CN 111481777 A CN111481777 A CN 111481777A CN 202010371109 A CN202010371109 A CN 202010371109A CN 111481777 A CN111481777 A CN 111481777A
- Authority
- CN
- China
- Prior art keywords
- subsystem
- virtual reality
- vein
- image processing
- glasses
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
- A61M5/00—Devices for bringing media into the body in a subcutaneous, intra-vascular or intramuscular way; Accessories therefor, e.g. filling or cleaning devices, arm-rests
- A61M5/42—Devices for bringing media into the body in a subcutaneous, intra-vascular or intramuscular way; Accessories therefor, e.g. filling or cleaning devices, arm-rests having means for desensitising skin, for protruding skin to facilitate piercing, or for locating point where body is to be pierced
- A61M5/427—Locating point where body is to be pierced, e.g. vein location means using ultrasonic waves, injection site templates
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0082—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence adapted for particular medical purposes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/48—Other medical applications
- A61B5/4887—Locating particular structures in or on the body
- A61B5/489—Blood vessels
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
Abstract
The invention relates to non-contact vein imaging virtual reality glasses based on memristor calculation, which comprise a near-infrared image acquisition subsystem, an image processing subsystem, a control motor subsystem and a virtual reality glasses imaging subsystem, wherein the near-infrared image acquisition subsystem, the image processing subsystem, the control motor subsystem and the virtual reality glasses imaging subsystem are sequentially connected. The invention discloses non-contact vein imaging virtual reality glasses, which combine an infrared image processing technology with visual information to perform image analysis and gradually realize an automatic vein imaging function through a dynamic sensing technology and an image enhancement technology. The efficiency of medical personnel seeking the vein is improved, the intravenous injection time of medical care is greatly shortened, the pain of a patient and medical disputes are reduced, and a vein data template can be formed to assist personal information comparison.
Description
Technical Field
The invention relates to virtual reality glasses, in particular to non-contact vein imaging virtual reality glasses based on memristor calculation.
Background
Venous access difficulties can have adverse effects on the patient, and multiple punctures increase anxiety and pain for the patient. Clinical studies have shown that 51% of children and 83% of children suffer from intense pain when receiving conventional venipuncture. Statistically, when a child in hospital sees the puncture needle tube, 74% of the pain and panic are caused by the difficulty of vein access; meanwhile, the difficulty in vein access can cause severe complications and multiple punctures, so that the trauma of a patient is increased, and meanwhile, the risk of phlebitis, venous infection and the like can be brought, and the additional cost is directly increased; in addition, the vein access is difficult to delay the rescue of patients, the vein puncture difficulty of children with the difficult vein access is extremely high, the average time for completing the puncture is 23min, in the period, the medicine can not enter the vein, the vein can also influence the throughput of the medicine, and therefore the optimal treatment time is delayed, and the rescue is delayed.
In the traditional intravenous injection process, in order to improve the success rate of venipuncture, a doctor adopts a method of patting the back of a hand and tightening arms to enable the vein prominence of a patient to be convenient to observe, but slight pain during patting and a tight pulse pressing belt bring trouble to the patient to a certain degree while the injection process of the doctor is increased; in addition, due to the large difference of vein distribution among patients, the method shows great limitation when patients with difficult vein access are encountered. Therefore, if a vein imaging method is available, pain can be overcome, and meanwhile vein patterns can be clearly displayed, the success rate of intravenous injection can be greatly improved, the injection time can be shortened, and the experience of patients can be improved.
Disclosure of Invention
In order to solve the problems of searching for venous vessels of infants, fat patients, beauty treatment patients and chemotherapy patients and accurately completing venipuncture, the invention provides non-contact type vein imaging virtual glasses, which are used for collecting image information of palms or arms of patients and users, transmitting the image information back to virtual reality glasses for display after algorithm segmentation and enhancement, and changing the head posture to control a motor to complete the movement of the visual field.
In order to achieve the purpose, the invention is realized by the following technical scheme:
the invention relates to non-contact vein imaging virtual reality glasses based on memristor calculation, which comprise a near-infrared image acquisition subsystem, an image processing subsystem, a control motor subsystem and a virtual reality glasses imaging subsystem, wherein the near-infrared image acquisition subsystem, the image processing subsystem, the control motor subsystem and the virtual reality glasses imaging subsystem are sequentially connected, and the glasses comprise:
the near-infrared image acquisition subsystem comprises a three-dimensional frame framework and a camera which form a system structure and are used for forming an image acquisition target area and acquiring near-infrared image information;
the image processing subsystem comprises an image processing module and a wireless transmission module connected with the control motor subsystem, the image processing module is used for filtering, segmenting, enhancing and identifying vein characteristic information in image data acquired by the near-infrared image acquisition subsystem, and the processed image information is provided for the virtual reality glasses visualization subsystem through the wireless transmission module;
the control motor subsystem comprises a motor control panel and a stepping motor and is used for processing the detection information of the attitude sensing module and realizing the function of controlling the motor to rotate corresponding shooting angles;
virtual reality glasses video picture subsystem includes VR glasses, gesture response module for the vein image information that the visual display conveys by the image processing subsystem, gesture response module passes through wireless transmission module and is connected with the control motor subsystem, is used for responding to user's head and rotates information, and with data transfer to the control motor subsystem.
The invention is further improved in that: the image processing module adopts a PYNQ-Z2 development board taking ZYNQ XC7Z020 FPGA as a core.
The invention is further improved in that: the convolution calculation of the image processing subsystem adopts a memristor matrix module.
The invention is further improved in that: the near-infrared image acquisition subsystem further comprises an 850nm optical filter and a light supplement plate, wherein the optical filter is used for filtering natural light and is combined with the light supplement plate to realize the function of preprocessing the acquired near-infrared image.
The invention has the beneficial effects that: the invention has simple operation, convenient use and no limitation of working time period; the invention can carry out focusing and light supplementing in full automation, and can realize near infrared image acquisition at the rate of 25 frames per second; the invention has stronger real-time performance, and can efficiently analyze, process and display the image information collected in the target area in real time; the invention has low cost, one-time development and multiple use, and is convenient for maintaining and modifying programs.
The invention combines the infrared image processing technology with the visual information to carry out image analysis and gradually realizes the automatic vein imaging function through the dynamic sensing technology and the image enhancement technology, thereby improving the vein searching efficiency of medical care personnel, greatly shortening the medical care intravenous injection time, reducing the pain of patients and the medical dispute, and also forming a vein data template to assist the personal information comparison.
Drawings
FIG. 1 is a flow chart of the distribution of the system of the present invention.
FIG. 2 is a flow chart of the algorithm of the image processing subsystem of the present invention.
FIG. 3 is a memristor cell resistance simulation circuit diagram of the present invention.
FIG. 4 is a diagram of a fundamental analog circuit of memristor convolution calculations in accordance with the present invention.
FIG. 5 is a flow chart of the RPN of the present invention.
FIG. 6 is a block diagram of the algorithm for controlling the motor subsystem of the present invention.
FIG. 7 is a system flow diagram of the present invention.
Detailed Description
In the following description, for purposes of explanation, numerous implementation details are set forth in order to provide a thorough understanding of the embodiments of the invention. It should be understood, however, that these implementation details are not to be interpreted as limiting the invention. That is, in some embodiments of the invention, such implementation details are not necessary. In addition, some conventional structures and components are shown in simplified schematic form in the drawings.
As shown in fig. 1 to 7, the invention relates to a non-contact vein imaging virtual reality glasses based on memristor calculation, the glasses system comprises a near-infrared image acquisition subsystem, an image processing subsystem, a control motor subsystem and a virtual reality glasses imaging subsystem, and the near-infrared image acquisition subsystem, the image processing subsystem, the control motor subsystem and the virtual reality glasses imaging subsystem are sequentially connected. The invention is used for solving the problem of efficiently searching for vein vessels, greatly shortening the medical intravenous injection time, and checking attendance of palm veins.
The near-infrared image acquisition subsystem comprises a three-dimensional frame framework and a camera which form a system structure, the three-dimensional frame is used as a basic frame of the system, and a target area for vein identification is selected; the camera is positioned in the center of the rectangular area of the central axis of the frame, so that the visual field of the camera can cover a lower target area through the rotation of the stepping motor, and the three-dimensional frame framework and the camera are used for forming an image acquisition target area and acquiring near-infrared image information; the near-infrared image acquisition subsystem further comprises an 850nm optical filter and a light supplement plate, wherein the optical filter is used for filtering natural light and is combined with the light supplement plate to realize the function of preprocessing the acquired near-infrared image;
the image processing subsystem comprises an image processing module, the image processing module is used for processing an arm image collected by the near-infrared image collecting subsystem and enhancing vein characteristic information in the image, the image processing module adopts a PYNQ-Z2 development board with ZYNQXC7Z020 FPGA as a core to perform enhancement processing, and a hardware memristor matrix module is adopted in convolution calculation, as shown in FIG. 2, the method mainly comprises four steps of image segmentation, noise filtering, characteristic enhancement and image identification:
(1) dividing the image, namely dividing and extracting the region of interest in the image by taking the target region as a boundary, specifically converting the image from a BGR color image into a gray image, searching coordinates of bottom end points of four corners of the bracket through Hough transformation, and cutting the coordinates through a resize function to obtain the target region;
(2) noise filtering, namely filtering background noise except for the contour of an arm, wherein a wiener filtering method is adopted in the invention, specifically, convolution operation is carried out on source image information and a specified degradation function, the convolution operation is completed by using a memristor matrix module, and the working principle of the memristor module is shown in FIGS. 3-4;
(3) enhancing the characteristics, namely enhancing the vein image by utilizing the characteristics of the arm vein image, wherein a histogram equalization algorithm is used in the method, specifically, a Hessian matrix is used as a reference, and a difference comparison method of characteristic vectors of the Hessian matrix is combined to perform Gaussian convolution on second-order integrals of the image;
(4) image recognition, namely effectively finishing the detection of the vein region with the characterization capability by using fast-RCNN, adopting CNN to extract the well-processed vein image characteristics in the invention, and then extracting the candidate vein characteristic region through region propofol, as shown in FIG. 5. The specific algorithm operation is as follows: feature extraction of the vein image is completed by adopting a VGG16, wherein the VGG16 comprises 13 convolutional layers, 13 activation layers and 4 pooling layers; the convolutional layer adopts relu as an activation function, the pooling layer adopts softmax as an activation function, and the RPN detects whether the candidate region contains vein information or not in the classification layer, and the candidate region has two output nodes which respectively represent the probability of belonging to the foreground, namely the vein or the background; in the regression layer, the feature detector is used for predicting the coordinates of the center point of the target candidate region respectively, and the specific coordinate information comprises a center coordinate point (x, y) and the length and width (w, h) of the candidate vein region.
The invention uses the intersection ratio (IOU) as the index for measuring the accuracy of the candidate area positioning, and the specific calculation formula is as follows:
IOU=(A∩B)/(A∪B)
a, B respectively represents a predicted candidate region and an actual vein region, the candidate region is represented by a four-dimensional vector [ x, y, w, h ], wherein x, y, w and h respectively represent the center coordinates of the candidate region and the width and height of the candidate region, the larger the IOU is, the closer the predicted candidate region is to a real calibration region, the more accurate the regression is shown, when the IOU of the predicted candidate region and the real vein region is smaller, the predicted bounding box needs to be adjusted, and the specific adjustment comprises translation and scale scaling.
Translational scaling
Scaling
Wherein the content of the first and second substances,
、
、
、
respectively, the scaling parameters obtained during the training phase. The error between the predicted candidate quiet region and the true venous region may be expressed as
Wherein, in the step (A),
the objective function is:
wherein the content of the first and second substances,
is a learnable parameter; ∅ (P)𝑖) A feature vector representing the input; n is the number of samples; d = x, y, w, h; and (4) performing target optimization by adopting a least square method.
The image processing subsystem further comprises a wireless transmission module connected with the control motor subsystem, and processed image information is provided for the virtual reality glasses visualization subsystem through the wireless transmission module.
The control motor subsystem comprises a motor control panel, a stepping motor and a universal joint and is used for processing detection information of the attitude sensing module and making a corresponding movement command for the motor control panel so as to realize the function of synchronous rotation of the visual field of a user and the visual field of the camera; as shown in FIG. 6, the present invention adopts a fuzzy PID algorithm to control the motor, and the fuzzy table is used to perform a certain degree of fuzzy to each attitude response, so as to reduce the error response of the system caused by uncontrollable or frequent angle changes.
Virtual reality glasses video picture subsystem includes wear-type VR glasses, gesture response module, and the user passes through VR glasses direct observation in the target area by the vein image information that the image processing subsystem conveys, and gesture response module passes through wireless transmission module and is connected with the control motor subsystem for response user's head rotates information, and with data transfer to the control motor subsystem.
The work flow of the system is shown in FIG. 7:
the power supply is turned on, the arm can be placed into a camera target detection area to observe after the virtual display glasses are arranged, the camera in the image acquisition subsystem acquires a near-infrared image, the acquired image is transmitted to the image processing subsystem to be divided, filtered, enhanced and identified, then the processed image is transmitted to the virtual display glasses, vein image observation is finally achieved, the head posture is changed, the posture induction module is triggered to transmit posture change information to the control motor subsystem to adjust the direction of the camera, and the function of synchronously adjusting the observation field of vision of a user is achieved.
The invention discloses non-contact vein imaging virtual reality glasses, which provide an arm vein image observation technology, realize direct observation of a processed vein image, transmit a target image of an observation target area in real time by a camera, divide extracted target information by an image processing module, enhance vein characteristics, transmit the processed image to the virtual reality glasses, and achieve the function of synchronizing visual fields by matching with an attitude sensing module.
The above description is only an embodiment of the present invention, and is not intended to limit the present invention. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims (4)
1. The utility model provides a non-contact vein visualization virtual reality glasses based on memristor is calculated which characterized in that: glasses include near-infrared image acquisition subsystem, image processing subsystem, control motor subsystem, virtual reality glasses video picture subsystem, near-infrared image acquisition subsystem the image processing subsystem control motor subsystem with virtual reality glasses video picture subsystem connects gradually, wherein:
the near-infrared image acquisition subsystem comprises a three-dimensional frame framework and a camera which form a system structure and are used for forming an image acquisition target area and acquiring near-infrared image information;
the image processing subsystem comprises an image processing module and a wireless transmission module connected with the control motor subsystem, the image processing module is used for filtering, segmenting, enhancing and identifying vein characteristic information in image data acquired by the near-infrared image acquisition subsystem, and the processed image information is provided for the virtual reality glasses visualization subsystem through the wireless transmission module;
the control motor subsystem comprises a motor control panel and a stepping motor and is used for processing the detection information of the attitude sensing module and realizing the function of controlling the motor to rotate corresponding shooting angles;
virtual reality glasses video picture subsystem includes VR glasses, gesture response module for the vein image information that the visual display conveys by the image processing subsystem, gesture response module passes through wireless transmission module and is connected with the control motor subsystem, is used for responding to user's head and rotates information, and with data transfer to the control motor subsystem.
2. The glasses according to claim 1, wherein the glasses comprise: the image processing module adopts a PYNQ-Z2 development board taking ZYNQ XC7Z020 FPGA as a core.
3. The glasses according to claim 1, wherein the glasses comprise: the convolution calculation of the image processing subsystem adopts a memristor matrix module.
4. The glasses according to claim 1, wherein the glasses comprise: the near-infrared image acquisition subsystem further comprises an 850nm optical filter and a light supplement plate, wherein the optical filter is used for filtering natural light and is combined with the light supplement plate to realize the function of preprocessing the acquired near-infrared image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010371109.1A CN111481777A (en) | 2020-05-06 | 2020-05-06 | Non-contact vein imaging virtual reality glasses based on memristor calculation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010371109.1A CN111481777A (en) | 2020-05-06 | 2020-05-06 | Non-contact vein imaging virtual reality glasses based on memristor calculation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111481777A true CN111481777A (en) | 2020-08-04 |
Family
ID=71790546
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010371109.1A Pending CN111481777A (en) | 2020-05-06 | 2020-05-06 | Non-contact vein imaging virtual reality glasses based on memristor calculation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111481777A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100823886B1 (en) * | 2006-12-08 | 2008-04-21 | 김성근 | Hypodermic vein detection imaging apparatus based on infrared optical system |
| US20150094662A1 (en) * | 2013-10-02 | 2015-04-02 | Korea Institute Of Science And Technology | Visualization apparatus for vein |
| CN104616260A (en) * | 2015-02-06 | 2015-05-13 | 武汉工程大学 | Vein image enhancement method and device |
| CN108143394A (en) * | 2017-12-27 | 2018-06-12 | 南京理工大学 | A kind of wearable portable vein imager |
| CN108363415A (en) * | 2018-03-29 | 2018-08-03 | 燕山大学 | A kind of vision remote control servomechanism and method applied to underwater robot |
-
2020
- 2020-05-06 CN CN202010371109.1A patent/CN111481777A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100823886B1 (en) * | 2006-12-08 | 2008-04-21 | 김성근 | Hypodermic vein detection imaging apparatus based on infrared optical system |
| US20150094662A1 (en) * | 2013-10-02 | 2015-04-02 | Korea Institute Of Science And Technology | Visualization apparatus for vein |
| CN104616260A (en) * | 2015-02-06 | 2015-05-13 | 武汉工程大学 | Vein image enhancement method and device |
| CN108143394A (en) * | 2017-12-27 | 2018-06-12 | 南京理工大学 | A kind of wearable portable vein imager |
| CN108363415A (en) * | 2018-03-29 | 2018-08-03 | 燕山大学 | A kind of vision remote control servomechanism and method applied to underwater robot |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jiang et al. | Gesture recognition based on skeletonization algorithm and CNN with ASL database | |
| CN107168527B (en) | The first visual angle gesture identification and exchange method based on region convolutional neural networks | |
| CN103838378B (en) | A kind of wear-type eyes control system based on pupil identification positioning | |
| CN107831900B (en) | human-computer interaction method and system of eye-controlled mouse | |
| CN106022213A (en) | Human body motion recognition method based on three-dimensional bone information | |
| CN109086754A (en) | A kind of human posture recognition method based on deep learning | |
| CN105159452B (en) | A kind of control method and system based on human face modeling | |
| Huan et al. | Deep convolutional neural networks for classifying body constitution based on face image | |
| CN111933275A (en) | Depression evaluation system based on eye movement and facial expression | |
| CN112669959B (en) | Automatic evaluation method for vitiligo conditions based on images | |
| CN112435731B (en) | Method for judging whether real-time gesture meets preset rules | |
| CN112597967A (en) | Emotion recognition method and device for immersive virtual environment and multi-modal physiological signals | |
| CN112232128B (en) | Eye tracking based method for identifying care needs of old disabled people | |
| Wu et al. | Appearance-based gaze block estimation via CNN classification | |
| CN110472546B (en) | Infant non-contact eye movement feature extraction device and method | |
| Leli et al. | Near-infrared-to-visible vein imaging via convolutional neural networks and reinforcement learning | |
| CN114202795A (en) | Method for quickly positioning pupils of old people | |
| Wu et al. | Anti-jamming heart rate estimation using a spatial–temporal fusion network | |
| CN112075981B (en) | Venipuncture robot control method, device and computer-readable storage medium | |
| Perez et al. | Real-time iris detection on coronal-axis-rotated faces | |
| CN111481777A (en) | Non-contact vein imaging virtual reality glasses based on memristor calculation | |
| CN111144331B (en) | Elbow vein image elbow median vein recognition method and elbow image acquisition device | |
| CN112162629A (en) | Real-time pupil positioning method based on circumscribed rectangle | |
| CN116749168A (en) | Rehabilitation track acquisition method based on gesture teaching | |
| CN114067422A (en) | Sight line detection method and device for driving assistance and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200804 |