CN117204958A - Image processing system and method based on fluorescent operation amplifying glasses - Google Patents

Image processing system and method based on fluorescent operation amplifying glasses Download PDF

Info

Publication number
CN117204958A
CN117204958A CN202211605757.4A CN202211605757A CN117204958A CN 117204958 A CN117204958 A CN 117204958A CN 202211605757 A CN202211605757 A CN 202211605757A CN 117204958 A CN117204958 A CN 117204958A
Authority
CN
China
Prior art keywords
image
fluorescent
glasses
amplifying
tumor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211605757.4A
Other languages
Chinese (zh)
Inventor
辛涛
万熠
王宏卫
王珊
韩敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
First Affiliated Hospital of Shandong First Medical University
Original Assignee
First Affiliated Hospital of Shandong First Medical University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by First Affiliated Hospital of Shandong First Medical University filed Critical First Affiliated Hospital of Shandong First Medical University
Priority to CN202211605757.4A priority Critical patent/CN117204958A/en
Publication of CN117204958A publication Critical patent/CN117204958A/en
Pending legal-status Critical Current

Links

Landscapes

  • Image Processing (AREA)

Abstract

The invention belongs to the technical field of image processing, and particularly relates to an image processing system and method based on fluorescent operation amplifying glasses, wherein the system comprises the following steps: the fluorescent operation amplifying glasses comprise glasses main bodies, and miniature headlamps, amplifying ocular lenses and miniature cameras which are arranged on the glasses main bodies; one end of the miniature head lamp is detachably connected with the optical filter; the miniature head lamp is used for providing different light sources according to the needs in the operation process; the miniature camera is used for acquiring an operation picture in an operation process; the fluorescent operation amplifying glasses perform data interaction with the image workstation; the image workstation is used for analyzing and processing the contrast image to realize automatic identification and calibration and space positioning of tumors or plaques. The invention can directly switch the operation picture and the contrast picture by adding the white light and the near infrared integrated miniature camera on the operation amplifying glasses and adding the head lamp to irradiate the white light or the light source with proper wavelength to the anatomical part.

Description

Image processing system and method based on fluorescent operation amplifying glasses
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to an image processing system and method based on fluorescent operation amplifying glasses.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
The operation magnifying glasses are widely used in surgical operations, such as cardiac surgery, ophthalmic surgery, microsurgery, plastic surgery, organ transplantation surgery, mammary gland repair surgery and the like, and vascular anastomosis, vascular bypass, lymphatic anastomosis and the like are all used for the operation magnifying glasses.
The operation magnifying glass is used as a conventional microscopic device for magnifying operation anatomical tissues and performing fine suturing operation. There are two configurations, one is 2.5 times, 3.5 times, or 4.5 times, 5 times magnifying lens barrel of wear-type adjustable pupil distance. The second high-end product is a custom glasses type magnifying glass, the magnification is 2 times to 5 times different, the existing operation magnifying glass adopts Galileo optical stereo convergence observation technology, binocular vision of an observer is converged in a narrow cavity in a stereo mode, a bright and amplified three-dimensional view field is generated, and a unique stereo deep layer scene is provided for inspection and treatment. The glasses, the reflector, the head lamp, the binocular magnifier and the like are integrated.
Near infrared fluorescence imaging (Near-infrared fluorescence imaging) is widely applied to the fields of medicine and biology by virtue of imaging sensitivity, rapidness and the like. The method is that a contrast agent which can emit fluorescence by the irradiation of the excitation light with near infrared wavelength is subjected to biological imaging, so as to reflect the physiological change of pathological tissues; but near infrared images only show the approximate outline of the target, the contrast of the target and the background is low, the edge is blurred, and details cannot be distinguished.
The evaluation of anastomosis effect of blood vessels and lymphatic vessels in operation is a blind area for operation under the operation amplifying glasses, and no technology can realize fluorescence radiography under the operation amplifying glasses and evaluate operation effect in operation through an image workstation. There are also techniques for tumor treatment that do not allow for real-time visualization under surgical magnifying glasses and for surgical protocol guidance.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides an image processing system and method based on fluorescent operation amplifying glasses.
To achieve the above object, one or more embodiments of the present invention provide the following technical solutions:
the first aspect of the invention provides an image processing system based on fluorescent operation amplifying glasses, which comprises the fluorescent operation amplifying glasses and an image workstation;
the fluorescent operation amplifying glasses comprise a glasses main body, and a miniature head lamp, an amplifying ocular and a miniature camera which are arranged on the glasses main body; one end of the miniature head lamp is detachably connected with the optical filter;
the glasses main body is a head-wearing glasses main body, and magnifying eyepieces are arranged on two sides of the glasses main body;
the middle upper part of the glasses main body is provided with a micro head lamp which is used for providing different light sources according to the needs in the operation process; a miniature camera is arranged below the miniature head lamp and is used for acquiring an operation image in an operation process;
the fluorescent operation amplifying glasses are in data interaction with the image workstation, the image workstation acquires operation images shot by the fluorescent operation amplifying glasses and analyzes and processes the operation images, and automatic identification, calibration and space positioning of tumors or plaques are achieved.
The second aspect of the invention provides an image processing method based on fluorescent operation amplifying glasses, comprising the following steps:
wearing fluorescent operation amplifying glasses, adjusting and adding a proper light source according to the requirement, injecting fluorescent contrast agent into a target blood vessel or tissue, and acquiring a color image and a fluorescent image by using a miniature camera;
transmitting the acquired color image and the fluorescent image to an image workstation;
the image workstation carries out identification and classification on the images;
and displaying the identified and classified tumor images on fluorescent operation magnifying glasses and an image workstation.
The one or more of the above technical solutions have the following beneficial effects:
(1) The invention can directly switch the operation picture and the contrast picture by adding the white light and the near infrared integrated miniature camera on the operation amplifying glasses and adding the head lamp to irradiate the white light or the light source with proper wavelength to the anatomical part. The fluorescent operation amplifying glasses are in communication connection with the image workstation, so that the purpose of operation picture storage and evaluation in contrast picture operation is achieved.
(2) According to the invention, the texture features and the depth features of the tumor region are respectively extracted, and a plurality of features are fused, and a tumor identification model is built by using a support vector mechanism, so that the automatic and accurate identification of tumor fluorescent images is effectively realized.
Additional aspects of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.
Fig. 1 is a schematic diagram of the structure of the fluorescent amplifying glasses of example 1.
Fig. 2 is a block diagram of an image processing subsystem according to embodiment 1.
Fig. 3 is an image processing flowchart of embodiment 2.
In the figure, 1, a glasses main body, 2, an optical filter, 3, a miniature head lamp, 4, an amplifying ocular, 5, a white light and near infrared light integrated miniature camera.
Detailed Description
The disclosure is further described below with reference to the drawings and examples.
Example 1
The invention provides an image processing system based on fluorescent operation amplifying glasses, comprising: fluorescent operation amplifying glasses, a power supply and an image workstation;
as shown in fig. 1, the fluorescent operation amplifying glasses comprise a glasses main body 1, a micro head lamp 3, an amplifying ocular 4 and a micro camera 5 are arranged on the glasses main body, a microprocessor, a communication module and an internal memory are arranged in the glasses main body, and the microprocessor is respectively connected with the communication module and the internal memory;
the glasses main body comprises a glasses frame and lenses, an amplifying ocular is arranged on the lenses, a nose pad is arranged between the two lenses, the miniature camera and the miniature head lamp are arranged on the nose pad through a bracket, the miniature head lamp is arranged above the miniature camera, and one end of the miniature head lamp is detachably connected with the optical filter;
the glasses main body is a head-mounted type and is embedded with an amplifying ocular, the amplifying ocular is amplified by 2.0-5.0 times, and the glasses main body is customized according to the needs of doctors and is used for realizing optical amplification on anatomical tissues in operation; the doctor directly observes the display screen of the image workstation through the operation magnifying eyepiece to realize the evaluation in operation, and the contrast image is simultaneously stored in the workstation, so that the later consulting, gear reserving and study are convenient; the optical filter can be selected as a 400 nanometer optical filter cover plate;
the miniature camera is a white light and near infrared integrated camera, can record an operation picture in real time under a white light source, and can switch to a near infrared mode to shoot an operation site injection contrast agent picture under a fluorescent light source; the internal memory is connected with the micro camera and used for storing the shot image, the microprocessor controls the communication module to perform data interaction with the image workstation, the stored image is sent to the image workstation, and the image processed by the image workstation is displayed through the fluorescent operation amplifying glasses;
the miniature head lamp is used for providing different light sources according to the needs in the operation process; the fluorescent operation amplifying glasses are used for acquiring an operation picture in an operation process and transmitting the operation picture to the image workstation; the image workstation is used for analyzing and processing the contrast image to realize automatic identification and calibration and space positioning of tumors or plaques.
The power output end of the power supply is connected with the micro head lamp and the micro camera of the operation amplifying glasses, the internal memory, the communication module, the processor and the like for supplying power, and a doctor hangs the operation amplifying glasses between the waist when using the operation amplifying glasses.
As shown in fig. 2, the image workstation is a computer, a tablet, a notebook or the like, and can realize the functions of storing, analyzing and processing images. The image workstation includes an image processing subsystem, the image processing subsystem including: the image preprocessing module is used for preprocessing the fluorescent image based on image denoising and contrast enhancement; the image feature extraction module is used for extracting features of the preprocessed image and extracting texture features and depth features of the image; the tumor classification and identification module is used for realizing automatic identification and calibration of tumors or plaques based on the trained tumor identification model; the tumor boundary superposition module is used for superposing the boundary outline of the tumor area into the color image; an image display module: the method is used for displaying the operation image and the identified image in the operation process.
Further, based on the fluorescence image, converting the fluorescence signal into a gray value generates a gray map image:
wherein I is an image containing a fluorescent signal, I min Is the minimum value of fluorescent signals in the image, I max For the maximum value of fluorescence signals in the image, round represents rounding the value to an integer, and I' is a gray mapping image;
based on the gray mapping image, noise reduction treatment is carried out on the image sequentially through wavelet transformation, median filtering and mean filtering;
automatically adjusting the contrast of the gray mapping image after the noise reduction treatment to obtain a gray image after the pretreatment,
I c =min(max(I’,C min ),C max );
wherein C is max For the set gray maximum value, C min For the set gray minimum value, I c For gray scale images, the min function represents taking the smaller of the two, the max function represents taking the larger of the two, I C Representing the preprocessed gray image;
further, in the image feature extraction module, multiple feature extraction is carried out on texture features and depth features of the preprocessed fluorescent image;
the texture features take tumor ROI area images of the preprocessed fluorescent images as extraction objects, and the extracted texture features comprise mean values, variances, skewness, peaks and energy;
the Mean (Mean) calculation formula is as follows:
wherein h is i For a pixel frequency of gray value i, G max The maximum gray value of the image;
the Variance (Variance) is calculated as follows:
the Skewness (Shewness) is calculated as follows:
the peak value (Kurtosis) is calculated as follows:
the Energy (Energy) is calculated as follows:
extracting depth features from the ROI region image, extracting deep learning features from the pooling layer by using a depth residual error network, and then performing normalization operation to obtain deep learning feature vectors;
further, in the tumor classification and identification module, a tumor identification model is built based on a Support Vector Machine (SVM), the tumor identification model is trained, a fluorescent image to be identified is input into the trained tumor identification model, and the identification of the tumor type in the fluorescent image is completed;
in the training process of the tumor recognition model, parameter optimization is carried out on the SVM; the SVM kernel function is a Gaussian radial kernel function, parameters C and g of the radial kernel function are automatically selected in the training process of the tumor recognition model, and the optimal parameters C and g are determined by a cross verification algorithm of a network search method for each different combination C and g while traversing the range C and g;
further, the boundary extraction module processes the extracted texture features and depth features by using a traditional digital image processing algorithm to obtain a boundary contour of the tumor, and the accurate positioning of the boundary of the tumor tissue is realized by distinguishing the tumor tissue from the healthy tissue;
considering that the near infrared image which is singly displayed only can display the rough outline of the tumor area, the problems of blurred edge and indistinguishable details are solved; the color image displayed alone cannot display the outline information of the tumor area;
in the embodiment, after the boundary of the tumor tissue is obtained, the boundary of the tumor tissue is superimposed on the color image and then displayed by the image workstation, so that a scientific basis is provided for a clinician to perform tumor resection operation.
Example two
The present embodiment provides an image processing method based on fluorescent operation amplifying glasses, based on an image processing system based on fluorescent operation amplifying glasses in the first embodiment, including:
wearing fluorescent operation amplifying glasses, adjusting and adding a proper light source according to the requirement, injecting contrast agent, and acquiring color images and fluorescent images by using a miniature camera;
specifically, after the fluorescent contrast agent is injected into a target blood vessel or tissue, the light source is adjusted to be a near infrared light source, and the fluorescent contrast agent is irradiated by the excitation light with near infrared wavelength to perform biological imaging, so that the physiological change of the lesion tissue is reflected; the color camera is matched with the LED light source, the near infrared light source is matched with the near infrared camera, and a color image of a tumor area of a patient and a near infrared optical image which cannot be seen by naked eyes are shot;
transmitting the acquired color image and the fluorescent image to an image workstation;
the image workstation carries out identification and classification on the images;
specifically, an image processing subsystem is operated in the image workstation to identify the fluorescent image, classify the tumor area and the non-tumor area, and superimpose the boundary outline of the tumor area on the color image, thereby providing scientific basis for the clinician to perform tumor resection operation;
displaying the identified and classified tumor images on fluorescent operation amplifying glasses and an image workstation;
specifically, by wearing the fluorescent operation amplifying glasses, not only can the shooting of the intra-operative images be realized in real time and the images are uploaded to the image processing workstation, medical staff observe the intra-operative images through the image processing workstation, but also the processed images are displayed by the fluorescent operation amplifying glasses, so that the simultaneous monitoring of a plurality of medical staff in the operation process is realized, and the comprehensive consultation of the illness state of a patient is facilitated.
While the foregoing description of the embodiments of the present invention has been presented in conjunction with the drawings, it should be understood that it is not intended to limit the scope of the invention, but rather, it is intended to cover all modifications or variations within the scope of the invention as defined by the claims of the present invention.

Claims (10)

1. An image processing system based on fluorescent operation amplifying glasses is characterized by comprising the fluorescent operation amplifying glasses and an image workstation;
the fluorescent operation amplifying glasses comprise a glasses main body, and a miniature head lamp, an amplifying ocular and a miniature camera which are arranged on the glasses main body; one end of the miniature head lamp is detachably connected with the optical filter;
the glasses main body is a head-wearing glasses main body, and magnifying eyepieces are arranged on two sides of the glasses main body;
the middle upper part of the glasses main body is provided with a micro head lamp which is used for providing different light sources according to the needs in the operation process; a miniature camera is arranged below the miniature head lamp and is used for acquiring an operation image in an operation process;
the fluorescent operation amplifying glasses are in data interaction with the image workstation, the image workstation acquires operation images shot by the fluorescent operation amplifying glasses and analyzes and processes the operation images, and automatic identification, calibration and space positioning of tumors or plaques are achieved.
2. The image processing system based on fluorescent operation magnifying glasses according to claim 1, wherein the micro-camera is a white light and near infrared integrated micro-camera, and is used for switching a white light mode or a near infrared mode according to requirements to record an operation picture.
3. The image processing system based on the fluorescent surgical magnifier glasses according to claim 1, further comprising a power supply, wherein the power supply is connected with the fluorescent surgical magnifier glasses for supplying power.
4. The image processing system based on fluorescent surgical magnifying glasses according to claim 1, wherein, the fluorescent operation amplifying glasses are worn by a doctor of a main knife, and when the middle blood vessel is anastomosed or the tumor operation is performed, injecting contrast agent into the operation anatomical part, coaxially matching light of the corresponding wave band of the contrast agent by using a miniature head lamp and an optical filter, displaying fluorescent images under an amplifying ocular lens and capturing and recording images by a red light camera after vascular fluorescent imaging.
5. The fluoroscopic surgical magnification glasses-based image processing system of claim 1 wherein the image workstation comprises an image processing subsystem comprising:
the image preprocessing module is used for preprocessing the fluorescent image based on image denoising and contrast enhancement;
the image feature extraction module is used for extracting features of the preprocessed image and extracting texture features and depth features of the image;
the tumor classification and identification module is used for realizing automatic identification and calibration of tumors or plaques based on the trained tumor identification model;
and the tumor boundary superposition module is used for superposing the boundary outline of the tumor area into the color image.
6. The image processing system based on fluorescent surgical magnifying glasses according to claim 5, wherein the preprocessing of the fluorescent image comprises: converting the fluorescent signal into a gray value based on the fluorescent image to generate a gray mapping image; and carrying out image noise reduction and contrast automatic adjustment based on the gray mapping image to obtain a preprocessed gray image.
7. The image processing system based on fluorescent surgical magnifying glasses according to claim 5, wherein the feature extraction of the preprocessed image comprises:
the texture features take a brain tumor ROI area image of the preprocessed gray level image as an extraction object, and the extracted texture features comprise mean, variance, skewness, peak value and energy.
8. The image processing system based on the fluorescent surgery amplifying glasses according to claim 5, wherein in the tumor classification and identification module, a tumor identification model is built based on a Support Vector Machine (SVM), the tumor identification model is trained, an image to be identified is input into the trained tumor identification model, and identification of tumor types in the fluorescent image is completed.
9. The fluorescence surgical magnifier-based image processing system according to claim 5, wherein said image workstation comprises an image display module for displaying surgical images during surgery and identified images.
10. An image processing method based on fluorescent operation amplifying glasses is characterized by comprising the following steps:
wearing fluorescent operation amplifying glasses, adjusting and adding a proper light source according to the requirement, injecting fluorescent contrast agent into a target blood vessel or tissue, and acquiring a color image and a fluorescent image by using a miniature camera;
transmitting the acquired color image and the fluorescent image to an image workstation;
the image workstation carries out identification and classification on the images;
and displaying the identified and classified tumor images on fluorescent operation magnifying glasses and an image workstation.
CN202211605757.4A 2022-12-14 2022-12-14 Image processing system and method based on fluorescent operation amplifying glasses Pending CN117204958A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211605757.4A CN117204958A (en) 2022-12-14 2022-12-14 Image processing system and method based on fluorescent operation amplifying glasses

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211605757.4A CN117204958A (en) 2022-12-14 2022-12-14 Image processing system and method based on fluorescent operation amplifying glasses

Publications (1)

Publication Number Publication Date
CN117204958A true CN117204958A (en) 2023-12-12

Family

ID=89034039

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211605757.4A Pending CN117204958A (en) 2022-12-14 2022-12-14 Image processing system and method based on fluorescent operation amplifying glasses

Country Status (1)

Country Link
CN (1) CN117204958A (en)

Similar Documents

Publication Publication Date Title
US11857317B2 (en) Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance
CN106236006B (en) 3D optical molecular image laparoscope imaging systems
CN108430306B (en) System and method for rapid inspection of vasculature and particulate flow using laser speckle contrast imaging
US8888284B2 (en) Field of light based device
EP3842689B1 (en) Method for adjusting surgical light parameters, surgical lighting device, and readable storage medium
KR101190265B1 (en) Head mouted operating magnifying apparatus
WO2015023990A1 (en) Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance
KR102267509B1 (en) The method for measuring microcirculation in cochlea and the apparatus thereof
JP6972049B2 (en) Image processing method and image processing device using elastic mapping of vascular plexus structure
KR20170016128A (en) Pen-type medical fluorescence image device and system which registers multiple fluorescent images using the same
CN108392181A (en) Region-of-interest tracks of device
CN112656369B (en) AR technology-based near-infrared fluorescent parathyroid gland identification enhancement method
JP2023534449A (en) Digital image optimization for ophthalmic surgery
CN108289606A (en) The resolution ratio enhancing of OCT image during vitrectomy
CN111603140B (en) In-situ visual positioning method and system for biological target
CN114300095A (en) Image processing apparatus, image processing method, image processing device, image processing apparatus, and storage medium
CN105962881A (en) Blood vessel recognition method and device
CN211131438U (en) Operation microscope system
WO2021256130A1 (en) Slit lamp microscope
CN108143501B (en) Anatomical projection method based on body surface vein features
CN112704473A (en) Parathyroid gland position enhancement equipment
CN107976795A (en) Binary channels tissue sample scanner and binary channels tissue sample digital imagery recurrence system
JP2022050738A (en) Slit lamp microscope system
CN117204958A (en) Image processing system and method based on fluorescent operation amplifying glasses
CN108392180A (en) Time-activity curve measurement device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination