CN110811495B - Focus size measuring method and system of fluorescence endoscope and computer equipment - Google Patents

Focus size measuring method and system of fluorescence endoscope and computer equipment Download PDF

Info

Publication number
CN110811495B
CN110811495B CN201911030178.XA CN201911030178A CN110811495B CN 110811495 B CN110811495 B CN 110811495B CN 201911030178 A CN201911030178 A CN 201911030178A CN 110811495 B CN110811495 B CN 110811495B
Authority
CN
China
Prior art keywords
fluorescence
focus
size
image
coordinate axis
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911030178.XA
Other languages
Chinese (zh)
Other versions
CN110811495A (en
Inventor
姚卫忠
李已晴
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Huanuokang Technology Co ltd
Original Assignee
Zhejiang Huanuokang Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Huanuokang Technology Co ltd filed Critical Zhejiang Huanuokang Technology Co ltd
Priority to CN201911030178.XA priority Critical patent/CN110811495B/en
Publication of CN110811495A publication Critical patent/CN110811495A/en
Application granted granted Critical
Publication of CN110811495B publication Critical patent/CN110811495B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/043—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances for fluorescence imaging
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00057—Operational features of endoscopes provided with means for testing or calibration
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10068—Endoscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Surgery (AREA)
  • Pathology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Optics & Photonics (AREA)
  • General Physics & Mathematics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • Theoretical Computer Science (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Geometry (AREA)
  • Endoscopes (AREA)

Abstract

The invention discloses a method, a system and computer equipment for measuring the size of a focus of a fluorescence endoscope, which are used for acquiring a fluorescence image and a visible light image of a diagnostic tissue and determining the size of a first focus on the fluorescence image according to the fluorescence image; under the condition that the fluorescence image and the visible light image are subjected to pixel alignment and image fusion, determining a second focus size on the visible light image according to the first focus size; determining the size of a third focus on a physical imaging plane according to the size of the second focus and the coordinate conversion matrix coefficient, and determining the imaging distance according to the functional relation among the fluorescence brightness, the excitation light intensity and the imaging distance of the fluorescence endoscope; and determining the actual size of the focus according to the third focus size and the imaging distance, thereby solving the problem that the applied fluorescence endoscope cannot quantitatively measure the actual size of the focus.

Description

Focus size measuring method and system of fluorescence endoscope and computer equipment
Technical Field
The application relates to the technical field of fluorescence endoscopes, in particular to a method and a system for measuring lesion size of a fluorescence endoscope and computer equipment.
Background
An endoscope is an important tool for examining and treating organs in a human body in modern medical treatment, and a fluorescence endoscope is a technology for marking lesion cells by using a fluorescence probe such as indocyanine green (ICG) and the like on the basis of a visible light endoscope and obtaining a focus image by using exciting light of other wave bands such as near infrared light and the like. However, in the related art, since the distance between the focus of the same size and the objective lens is different, and the size of the focus image displayed on the image display is also different, the actual size of the focus cannot be quantitatively obtained only according to the size of the image of the focus on the display, and in the process of diagnosis or surgery, a doctor is usually required to subjectively estimate the actual size of the focus by experience, and the judgment error thereof may affect the diagnosis result.
Aiming at the problem that the fluorescence endoscope cannot quantitatively measure the actual size of the focus in the related technology, no effective solution is provided at present.
Disclosure of Invention
The invention provides a method, a system and a computer device for measuring the lesion size of a fluorescence endoscope, aiming at solving the problem that the fluorescence endoscope cannot quantitatively measure the actual size of the lesion in the related art.
According to one aspect of the present invention, there is provided a fluorescence endoscope-based lesion size measurement system, the system comprising a fluorescence endoscope and a control device;
the fluorescence endoscope acquires a fluorescence image and a visible light image of a diagnostic tissue, and the control device determines a first lesion size on the fluorescence image according to the fluorescence image;
the control device determines a second lesion size on the visible light image according to the first lesion size under the condition of performing pixel alignment and image fusion on the fluorescence image and the visible light image;
the control device determines the size of a third focus on a physical imaging plane according to the size of the second focus and the coordinate conversion matrix coefficient, and determines the imaging distance according to the functional relation among the fluorescence brightness, the excitation light intensity and the imaging distance of the fluorescence endoscope; the pixel plane of the visible light image is fixed on the physical imaging plane, and the coordinate conversion matrix coefficient is obtained according to the calibration of a camera corresponding to the fluorescence endoscope;
the control device determines the actual size of the focus according to the third focus size and the imaging distance.
In one embodiment, the control device is further configured to acquire physical parameters of the fluorescence endoscope, wherein the physical parameters include a prism parameter, a first focal length and a second focal length;
the control device obtains the angle relation among the prism parameter, the first refraction angle and the second refraction angle according to the refraction law; wherein the first refraction angle is a refraction angle on a visible light imaging target surface of the fluorescence endoscope, and the second refraction angle is a refraction angle on a near infrared light imaging target surface of the fluorescence endoscope;
and the control device determines the size of the second focus according to the angle relationship, the first focus, the second focus and the size of the first focus, wherein the first focus is the focus of the visible light imaging target surface, and the second focus is the focus of the near infrared light imaging target surface.
In one embodiment, the system further comprises a statistical device, wherein the statistical device is configured to obtain a fluorescence brightness statistic of the fluorescence endoscope at different imaging distances and different excitation light intensities, and fit the functional relationship according to the fluorescence brightness statistic, and the control device determines the imaging distance according to the functional relationship.
In one embodiment, the control device is further configured to obtain a first number of pixels on a first coordinate axis and a second number of pixels on a second coordinate axis of the first lesion size from the fluorescence image; wherein the first coordinate axis is perpendicular to the second coordinate axis;
the control device acquires unit pixel information of the fluorescence image according to the resolution of the fluorescence image and the target surface parameter of the fluorescence endoscope; wherein the unit pixel information includes first unit pixel information on the first coordinate axis and second unit pixel information on the second coordinate axis;
the control device acquires a first length value of the first lesion size on the first coordinate axis according to the first pixel number and the first unit pixel information; and the control device acquires a second length value of the first lesion size on the second coordinate axis according to the second pixel number and the second unit pixel information.
According to another aspect of the present invention, there is provided a lesion size measuring method of a fluorescence endoscope, the method including:
acquiring a fluorescence image and a visible light image of a diagnostic tissue, and determining a first lesion size on the fluorescence image according to the fluorescence image;
under the condition that the fluorescence image and the visible light image are subjected to pixel alignment and image fusion, determining a second focus size on the visible light image according to the first focus size;
determining the size of a third focus on a physical imaging plane according to the size of the second focus and the coordinate conversion matrix coefficient, and determining the imaging distance according to the functional relation among the fluorescence brightness, the excitation light intensity and the imaging distance of the fluorescence endoscope; the pixel plane of the visible light image is fixed on the physical imaging plane, and the coordinate conversion matrix coefficient is obtained according to the calibration of a camera corresponding to the fluorescence endoscope;
and determining the actual size of the focus according to the third focus size and the imaging distance.
In one embodiment, the determining a second lesion size on the visible light image based on the first lesion size in the case of performing pixel alignment and image fusion on the fluorescence image and the visible light image comprises:
acquiring physical parameters of the fluorescence endoscope, wherein the physical parameters comprise prism parameters, a first focal length and a second focal length;
acquiring the angle relation among the prism parameters, the first refraction angle and the second refraction angle according to the refraction law; wherein the first refraction angle is a refraction angle on a visible light imaging target surface of the fluorescence endoscope, and the second refraction angle is a refraction angle on a near infrared light imaging target surface of the fluorescence endoscope;
and determining the size of the second focus according to the angle relation, the first focus, the second focus and the size of the first focus, wherein the first focus is the focus of the visible light imaging target surface, and the second focus is the focus of the near infrared light imaging target surface.
In one embodiment, the determining the imaging distance according to the functional relationship among the fluorescence brightness, the excitation light intensity and the imaging distance of the fluorescence endoscope comprises:
and acquiring fluorescence brightness statistics of the fluorescence endoscope under different imaging distances and different excitation light intensities, fitting the functional relation according to the fluorescence brightness statistics, and determining the imaging distance according to the functional relation.
In one embodiment, the determining the first lesion size on the fluorescence image from the fluorescence image comprises:
acquiring a first pixel number on a first coordinate axis and a second pixel number on a second coordinate axis of the first lesion size according to the fluorescence image; wherein the first coordinate axis is perpendicular to the second coordinate axis;
acquiring unit pixel information of the fluorescence image according to the resolution of the fluorescence image and the target surface parameter of the fluorescence endoscope; wherein the unit pixel information includes first unit pixel information on the first coordinate axis and second unit pixel information on the second coordinate axis;
acquiring a first length value of the first lesion size on the first coordinate axis according to the first pixel number and the first unit pixel information; and acquiring a second length value of the first lesion size on the second coordinate axis according to the second pixel number and the second unit pixel information.
According to another aspect of the present invention, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of any of the methods described above when executing the computer program.
According to another aspect of the invention, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that the computer program realizes the steps of any of the methods described above when executed by a processor.
According to the invention, a fluorescence image and a visible light image of a diagnostic tissue are obtained by adopting a focus size measuring method, a system and computer equipment of a fluorescence endoscope, and the size of a first focus on the fluorescence image is determined according to the fluorescence image; under the condition that the fluorescence image and the visible light image are subjected to pixel alignment and image fusion, determining a second focus size on the visible light image according to the first focus size; determining the size of a third focus on a physical imaging plane according to the size of the second focus and the coordinate conversion matrix coefficient, and determining the imaging distance according to the functional relation among the fluorescence brightness, the excitation light intensity and the imaging distance of the fluorescence endoscope; and determining the actual size of the focus according to the third focus size and the imaging distance, thereby solving the problem that the applied fluorescence endoscope cannot quantitatively measure the actual size of the focus.
Drawings
FIG. 1 is a schematic illustration of a related art fluorescence endoscope, according to an embodiment of the present invention;
FIG. 2 is a first flowchart of a lesion size measurement method according to an embodiment of the present invention;
FIG. 3 is a first diagram illustrating a transformation model according to an embodiment of the invention;
FIG. 4 is a flowchart illustrating a lesion size measurement method according to an embodiment of the present invention;
FIG. 5 is a second diagram of a transformation model according to an embodiment of the invention;
FIG. 6 is a schematic view of a lesion under a visible light image according to an embodiment of the present invention;
FIG. 7 is a flowchart of a lesion size measuring method according to an embodiment of the present invention;
FIG. 8 is a fourth flowchart of a lesion size measurement method according to an embodiment of the present invention;
FIG. 9 is a schematic view of a lesion under a fluorescence image according to an embodiment of the present invention;
FIG. 10 is a block diagram of a lesion size measurement system according to an embodiment of the present invention;
fig. 11 is a block diagram of a lesion size measurement system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In the present embodiment, a fluorescence endoscope in the related art is provided, and fig. 1 is a schematic diagram of a fluorescence endoscope in the related art according to an embodiment of the present invention, and as shown in fig. 1, the basic structure of the fluorescence endoscope includes a lens tube 12, a lens 14, a beam splitter prism 16, a first optical filter 18, a visible light sensor 110, a second optical filter 112, and a near infrared sensor 114: the imaging system uses a camera, the microscope tube 12 detects a diagnostic tissue and shoots the tissue through the lens 14, light supplement is performed at the position A, a supplemented light source couples visible light and near infrared light, the mixed light can be separated through the beam splitter prism 16, wherein the visible light reaches the visible light sensor 110 through the first optical filter 18, and the near infrared light reaches the near infrared sensor 114 through the second optical filter 112, so that a color image and a fluorescence image at the same position are obtained respectively.
In this embodiment, a lesion size measuring method of a fluorescence endoscope is provided, and fig. 2 is a flowchart of a lesion size measuring method according to an embodiment of the present invention, as shown in fig. 2, the method includes the following steps:
step S202, acquiring a fluorescence image and a visible light image of a diagnostic tissue, and determining a first focus size on the fluorescence image according to the fluorescence image; after the fluorescence endoscope is inserted into the human body or the animal body, the fluorescence image and the visible light image can be simultaneously obtained, the near-infrared sensor 114 obtains the fluorescence image in real time, and the visible light sensor 110 obtains the visible light image in real time; the fluorescence image does not contain color information, but the focus part can be reflected under the excitation of near infrared light due to the existence of ICG marks, namely, the brightness of a lesion part is higher than that of a normal part, so that a highlight area of the fluorescence image is a marked focus pixel plane range, and the first focus size can be obtained by calculating according to the focus pixel plane range.
Step S204, under the condition of carrying out pixel alignment and image fusion on the fluorescence image and the visible light image, determining a second focus size on the visible light image according to the first focus size; wherein the image fusion is performed after the pixel alignment so that the fluorescence image is synthesized with the visible light image to form a color image indicating the lesion, and the pixel plane size of the lesion region of the visible light image is obtained.
Step S206, determining the size of a third focus on a physical imaging plane according to the size of the second focus and the coordinate transformation matrix coefficient, and determining the imaging distance according to the functional relation among the fluorescence brightness, the excitation light intensity and the imaging distance of the fluorescence endoscope; the pixel plane of the visible light image is fixed on the physical imaging plane, and the coordinate conversion matrix coefficient is obtained according to camera calibration; the coordinate transformation matrix coefficient is a transformation parameter K from the visible light pixel plane to the physical imaging plane, and as shown in formula 1 and formula 2, the third lesion size can be calculated:
L_a(x,y)＝K×A1B1equation 1
L_b(x,y)＝K×A1C1Equation 2
Wherein A is1B1Is the length of the second lesion size on the first axis, A1C1Is the length of the second lesion size on the second axis, L _ a (x, y) is the length of the third lesion size on the first axis, and L _ b (x, y) is the length of the third lesion size on the second axis.
Step S208, determining the actual size of the focus according to the third focus size and the imaging distance; fig. 3 is a first schematic diagram of a conversion model according to an embodiment of the present invention, as shown in fig. 3, where a rectangle represents a beam splitter prism, EF is a refraction surface, O point is an equivalent optical center obtained by calibrating a fluorescence endoscope system, and L is a focus of a physical imaging plane on a first coordinate axis; according to the pinhole imaging principle, the actual range of the objective world focus can be obtained by the first model in the figure 3; since the pixel plane is fixed on the physical imaging plane, the distance from the equivalent optical center O to the physical imaging plane is equal to the distance from the pixel plane, and both are the first focal length f1The first focal length is the focal length of the visible light imaging target surface, and the actual size of the focus is determinedAs shown in equations 3 and 4:
Figure BDA0002249914450000061
Figure BDA0002249914450000062
wherein D is the imaging distance, AB is the length of the actual size of the lesion on the first axis, and AC is the length of the actual size of the lesion on the second axis.
The actual size of the lesion in the related art generally requires a doctor to subjectively estimate by experience, and the embodiment of the present invention passes through the above-described steps S202 to S208, on the basis of marking the focus by a fluorescence imaging technology, the imaging distance is determined by the functional relation of fluorescence brightness, excitation light intensity and the imaging distance of the fluorescence endoscope, then the actual size of the focus is quantitatively calculated according to the registration between the fluorescence image, the visible light image and the actual focus, the optical geometric relation and the imaging distance, the distance is determined without contacting the focus and repeatedly moving the endoscope position, the operation burden of a doctor is reduced, the problem that the fluorescence endoscope cannot quantitatively measure the actual size of the focus is solved, in addition, the visible light image and the fluorescence image can be obtained simultaneously, and the error of focus size measurement caused by micro movement in the light switching process is avoided.
In one embodiment, a lesion size measuring method of a fluorescence endoscope is provided, and fig. 4 is a flowchart illustrating a lesion size measuring method according to an embodiment of the present invention, as shown in fig. 4, the method includes the following steps:
step S402, acquiring physical parameters of the fluorescence endoscope, wherein the physical parameters comprise prism parameters, a first focal length and a second focal length; the prism parameters are recorded as
Figure BDA0002249914450000072
Namely, the included angle between the refraction surface and the edge of the beam splitter prism 16; the first focal length and the second focal lengthThe distance is obtained by calibrating a fluorescence endoscope system.
Step S404, acquiring the angle relation among the prism parameter, the first refraction angle and the second refraction angle according to the refraction law; the first refraction angle is a refraction angle on a visible light imaging target surface of the fluorescence endoscope, and the second refraction angle is a refraction angle on a near infrared light imaging target surface of the fluorescence endoscope; FIG. 5 is a second schematic diagram of a conversion model according to an embodiment of the invention, as shown in FIG. 5, wherein a rectangle represents the beam splitter prism 16, EF is a refractive surface, and SO is a normal perpendicular to the refractive surface; among them, the law of refraction can be derived:
∠AOB＝∠A2OB2＝∠A1OB1α equation 5
Figure BDA0002249914450000071
Step S406, determining the size of the second focus according to the angle relationship, the first focus, the second focus and the size of the first focus, wherein the first focus is the focus of the visible light imaging target surface, and the second focus is the focus of the near infrared light imaging target surface;
marker B1OH1β, according to equation 6:
Figure BDA0002249914450000081
Figure BDA0002249914450000082
wherein, B2H2B can be calculated for the fluorescence image pixel plane distance by combining formula 7 and formula 81H1Length value of when
Figure BDA0002249914450000085
The calculation can be simplified; similarly, according to equation 5, we can obtain:
Figure BDA0002249914450000083
Figure BDA0002249914450000084
a can be calculated by combining the formulas 10 and 111H1A length value.
FIG. 6 is a schematic view of a lesion under a visible light image according to an embodiment of the present invention, as shown in FIG. 6, A1B1For the length of the second lesion size on the first axis, A can be obtained from the above formula1B1As shown in equation 11:
A1B1＝A1H1-B1H1equation 11
The length of the second lesion size on the second coordinate axis may be obtained in the same manner.
Through the steps S402 to S406, in the process of pixel alignment and image fusion, the corresponding relationship between the fluorescence image and the visible light image is optimized according to the conversion model, so that the pixel range of the focus of the visible light image is more accurate, and the accuracy of quantitatively measuring the actual size of the focus is further improved.
In one embodiment, a lesion size measuring method of a fluorescence endoscope is provided, and fig. 7 is a flowchart illustrating a third method of measuring a lesion size according to an embodiment of the present invention, as shown in fig. 7, the method including the steps of:
step S702, obtaining the fluorescence brightness statistic value of the fluorescence endoscope under different imaging distances and different excitation light intensities, fitting the function relation according to the fluorescence brightness statistic value, and determining the imaging distance according to the function relation; wherein, the imaging distance of fluorescence image and excitation light intensity can influence the luminance value of fluorescence, carry out the human external experiment, record fluorescence luminance statistic, the imaging distance, the change data of excitation light intensity, and according to this change data fitting that acquire luminance statistic Y, the functional relationship between imaging distance D and the excitation light intensity I, this functional relationship is marked as Y ═ F (D, I), current excitation light intensity I can be surveyed by this fluorescence endoscope, current fluorescence luminance statistic Y can be obtained by statistical information, consequently according to functional relationship F, can solve current imaging distance D.
Through the step S702, the functional relationship among the luminance statistic, the imaging distance and the excitation light intensity is fitted according to the fluorescence luminance statistic of the fluorescence endoscope obtained by the experiment under different imaging distances and different excitation light intensities, and the imaging distance is determined according to the functional relationship, without the need of moving the endoscope or an additional hardware unit for multiple times to realize distance measurement, so that the error caused in the moving process is reduced, and the operation burden of a doctor is reduced.
In one embodiment, a lesion size measuring method of a fluorescence endoscope is provided, and fig. 8 is a flowchart illustrating a lesion size measuring method according to an embodiment of the present invention, as shown in fig. 8, the method including the steps of:
step S802, acquiring a first pixel number on a first coordinate axis and a second pixel number on a second coordinate axis of the first lesion size according to the fluorescence image; the first coordinate axis is vertical to the second coordinate axis; in this embodiment, the number of the first pixels is n, and the number of the second pixels is m; fig. 9 is a schematic view of a lesion under a fluorescence image according to an embodiment of the present invention, and as shown in fig. 9, a highlight portion in the fluorescence image is a lesion region marked by a fluorescence probe, and the number n and m of pixels at the longest and widest portions of the lesion can be obtained by simply calculating fluorescence image data of an endoscope.
Step S804, acquiring unit pixel information of the fluorescence image according to the resolution of the fluorescence image and the target surface parameter of the fluorescence endoscope; wherein the unit pixel information includes first unit pixel information on the first coordinate axis and second unit pixel information on the second coordinate axis; according to the data manual, the target surface width and height of the near infrared light sensor are r _ a and r _ b respectively, the resolution of the shot image is r _ width × r _ height, the first coordinate axis is an x-axis, the second coordinate axis is a y-axis, and then the lengths of each pixel in the x-axis and y-axis directions under the pixel coordinate system of the fluorescence image are respectively:
Figure BDA0002249914450000091
step S806, obtaining a first length value of the first lesion size on the first coordinate axis according to the first pixel number and the first unit pixel information; acquiring a second length value of the first lesion size on the second coordinate axis according to the second pixel number and the second unit pixel information; through the foregoing steps, A2B2、A2C2The length at the fluorescence image pixel plane is shown in equations 13 and 14:
A2B2formula 13 of nxr _ dx
A2C2Equation 14 of mxr _ dy
Wherein A is2B2Is a first length value of the first lesion size on the first coordinate axis, A2C2Is a second length value of the first lesion size on the second coordinate axis.
Through the above steps S802 to S806, the length information of the first lesion size is obtained through the statistical information of the fluorescence image, and the near infrared light can directly track the lesion marked by the fluorescence probe without using image segmentation or other techniques to obtain the lesion information.
It should be understood that, although the steps in the flowcharts of fig. 2, 4, 7 and 8 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2, 4, 7, and 8 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternatingly with other steps or at least some of the sub-steps or stages of other steps.
In the present embodiment, a lesion size measuring system of a fluorescence endoscope is provided, and fig. 10 is a block diagram showing a structure of a lesion size measuring system according to an embodiment of the present invention, as shown in fig. 10, the system includes a fluorescence endoscope 102 and a control device 104;
the fluorescence endoscope 102 acquires a fluorescence image and a visible light image of a diagnostic tissue, and the control device 104 determines a first lesion size on the fluorescence image according to the fluorescence image;
the control device 104 determines a second lesion size on the visible light image according to the first lesion size when performing pixel alignment and image fusion on the fluorescence image and the visible light image;
the control device 104 determines the third lesion size on the physical imaging plane according to the second lesion size and the coordinate transformation matrix coefficient, and determines the imaging distance according to the functional relationship among the fluorescence brightness, the excitation light intensity and the imaging distance of the fluorescence endoscope 102; wherein, the pixel plane of the visible light image is fixed on the physical imaging plane, and the coordinate transformation matrix coefficient is obtained according to the camera calibration corresponding to the fluorescence endoscope 102;
the control device 104 determines an actual size of the lesion based on the third lesion size and the imaging distance.
Through the above embodiment, the fluorescence endoscope 102 acquires the fluorescence image and the visible light image of the diagnostic tissue, the imaging distance is determined through the functional relationship among the fluorescence brightness, the excitation light intensity and the imaging distance of the fluorescence endoscope 102 on the basis of marking the focus by the fluorescence imaging technology, the control device 104 quantitatively calculates the actual size of the focus according to the registration, the optical geometric relationship and the imaging distance among the fluorescence image, the visible light image and the actual focus, the focus is not required to be contacted and the endoscope position is not required to be repeatedly moved to determine the distance, the operation burden of a doctor is reduced, and the problem that the fluorescence endoscope 104 cannot quantitatively measure the actual size of the focus is solved.
In one embodiment, the control device 104 of the lesion size measurement system is further configured to acquire physical parameters of the fluorescence endoscope 102, wherein the physical parameters include a prism parameter, a first focal length, and a second focal length;
the control device 104 obtains the angle relationship among the prism parameter, the first refraction angle and the second refraction angle according to the law of refraction; wherein the first refraction angle is a refraction angle on the visible light imaging target surface of the fluorescence endoscope 102, and the second refraction angle is a refraction angle on the near-infrared light imaging target surface of the fluorescence endoscope 102;
the control device 104 determines the second focal length according to the angular relationship, the first focal length, the second focal length, and the first focal length, wherein the first focal length is the focal length of the visible light imaging target surface, and the second focal length is the focal length of the near-infrared light imaging target surface.
In one embodiment, a lesion size measuring system of a fluorescence endoscope is provided, fig. 11 is a block diagram of a lesion size measuring system according to an embodiment of the present invention, as shown in fig. 11, the system further includes a statistical device 112;
the statistical device 112 is configured to obtain a fluorescence brightness statistic of the fluorescence endoscope 102 at different imaging distances and different excitation light intensities, fit the functional relationship according to the fluorescence brightness statistic, and determine the imaging distance according to the functional relationship by the control device 104.
In one embodiment, the control means 104 of the lesion size measurement system is further configured to obtain a first number of pixels on a first coordinate axis and a second number of pixels on a second coordinate axis of the first lesion size from the fluorescence image; the first coordinate axis is vertical to the second coordinate axis;
the control device 104 acquires the unit pixel information of the fluorescence image according to the resolution of the fluorescence image and the target surface parameters of the fluorescence endoscope 102; wherein the unit pixel information includes first unit pixel information on the first coordinate axis and second unit pixel information on the second coordinate axis;
the control device 104 obtains a first length value of the first lesion size on the first coordinate axis according to the first pixel number and the first unit pixel information; the control device 104 obtains a second length value of the first lesion size on the second coordinate axis according to the second pixel number and the second unit pixel information.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (9)

1. A fluorescence endoscope-based lesion size measurement system, the system comprising: a fluorescence endoscope and control device;
the fluorescence endoscope acquires a fluorescence image and a visible light image of a diagnostic tissue, and the control device determines a first lesion size on the fluorescence image according to the fluorescence image;
the control device determines a second lesion size on the visible light image according to the first lesion size under the condition of performing pixel alignment and image fusion on the fluorescence image and the visible light image;
the control device determines the size of a third focus on a physical imaging plane according to the size of the second focus and the coordinate conversion matrix coefficient, and determines the imaging distance according to the functional relation among the fluorescence brightness, the excitation light intensity and the imaging distance of the fluorescence endoscope; the pixel plane of the visible light image is fixed on the physical imaging plane, and the coordinate conversion matrix coefficient is obtained according to the calibration of a camera corresponding to the fluorescence endoscope;
the control device determines the actual size of the focus according to the third focus size and the imaging distance.
2. The system of claim 1, wherein the control device is further configured to acquire physical parameters of the fluorescence endoscope, wherein the physical parameters include a prism parameter, a first focal length, and a second focal length;
the control device obtains the angle relation among the prism parameter, the first refraction angle and the second refraction angle according to the refraction law; wherein the first refraction angle is a refraction angle on a visible light imaging target surface of the fluorescence endoscope, and the second refraction angle is a refraction angle on a near infrared light imaging target surface of the fluorescence endoscope;
and the control device determines the size of the second focus according to the angle relationship, the first focus, the second focus and the size of the first focus, wherein the first focus is the focus of the visible light imaging target surface, and the second focus is the focus of the near infrared light imaging target surface.
3. The system of claim 1, further comprising a statistical device, wherein the statistical device is configured to obtain fluorescence intensity statistics of the fluorescence endoscope at different imaging distances and different excitation intensities, and to fit the functional relationship according to the fluorescence intensity statistics, and the control device determines the imaging distances according to the functional relationship.
4. The system of claim 1, wherein the control device is further configured to obtain a first number of pixels on a first coordinate axis and a second number of pixels on a second coordinate axis of the first lesion size from the fluorescence image; wherein the first coordinate axis is perpendicular to the second coordinate axis;
the control device acquires unit pixel information of the fluorescence image according to the resolution of the fluorescence image and the target surface parameter of the fluorescence endoscope; wherein the unit pixel information includes first unit pixel information on the first coordinate axis and second unit pixel information on the second coordinate axis;
the control device acquires a first length value of the first lesion size on the first coordinate axis according to the first pixel number and the first unit pixel information; and the control device acquires a second length value of the first lesion size on the second coordinate axis according to the second pixel number and the second unit pixel information.
5. The system of claim 1, wherein the third lesion size is determined according to the following formula:
L_a(x,y)＝K×A1B1
L_b(x,y)＝K×A1C1
where K is the coordinate transformation matrix coefficient, A1B1Is the length of the second lesion size on the first coordinate axis, A1C1Is the length of the second lesion size on a second coordinate axis, L _ a (x, y) is the length of the third lesion size on the first coordinate axis, and L _ b (x, y) is the length of the third lesion size on the second coordinate axis.
6. The system of claim 5, wherein the lesion actual size is determined according to the following formula:
Figure FDA0003373443300000021
Figure FDA0003373443300000022
wherein, the O point is the equivalent optical center, L is the focus of the physical imaging plane on the first coordinate axis, f1Is a first focal length, D is the imaging distance, AB is the length of the lesion actual size on the first coordinate axis, and AC is the length of the lesion actual size on the second coordinate axis.
7. The system of claim 1, wherein the fluorescence endoscope comprises a near infrared sensor and a visible light sensor; the near-infrared sensor is used for acquiring the fluorescence image, and the visible light sensor is used for acquiring the visible light image.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor, when executing the computer program, performs the operations of the lesion size measurement system of any of claims 1-7.
9. A computer-readable storage medium having stored thereon a computer program, characterized in that when the computer program is executed, the operation of the lesion size measurement system according to any one of the preceding claims 1 to 7 is performed.
CN201911030178.XA 2019-10-28 2019-10-28 Focus size measuring method and system of fluorescence endoscope and computer equipment Active CN110811495B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911030178.XA CN110811495B (en) 2019-10-28 2019-10-28 Focus size measuring method and system of fluorescence endoscope and computer equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911030178.XA CN110811495B (en) 2019-10-28 2019-10-28 Focus size measuring method and system of fluorescence endoscope and computer equipment

Publications (2)

Publication Number Publication Date
CN110811495A CN110811495A (en) 2020-02-21
CN110811495B true CN110811495B (en) 2022-04-05

Family

ID=69550939

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911030178.XA Active CN110811495B (en) 2019-10-28 2019-10-28 Focus size measuring method and system of fluorescence endoscope and computer equipment

Country Status (1)

Country Link
CN (1) CN110811495B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115381389A (en) * 2021-05-24 2022-11-25 山东威高宏瑞医学科技有限公司 System and method for measuring absolute size of focus under endoscope
CN113376931B (en) * 2021-06-10 2022-11-04 复旦大学 Fluorescence area light intensity supplementary lighting method, device and system for parathyroid gland detection
CN114882093B (en) * 2022-07-12 2022-10-28 广东欧谱曼迪科技有限公司 Intraoperative three-dimensional point cloud generation method and device and intraoperative local structure navigation method
CN114882096B (en) * 2022-07-12 2023-05-16 广东欧谱曼迪科技有限公司 Method and device for measuring distance under fluorescent endoscope, electronic equipment and storage medium
CN117064311B (en) * 2023-10-16 2024-01-30 深圳迈瑞生物医疗电子股份有限公司 Endoscopic image processing method and endoscopic imaging system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06229931A (en) * 1993-02-08 1994-08-19 Olympus Optical Co Ltd Method and equipment for calibration for image measuring apparatus
JPH07281105A (en) * 1994-02-21 1995-10-27 Olympus Optical Co Ltd Endoscope device
CN106361281A (en) * 2016-08-31 2017-02-01 北京数字精准医疗科技有限公司 Fluorescent real-time imaging and fusing method and device
CN109730683A (en) * 2018-12-21 2019-05-10 重庆金山医疗器械有限公司 Endoscope object size calculation method and analysis system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006006834A (en) * 2004-06-29 2006-01-12 Pentax Corp Electronic endoscope system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06229931A (en) * 1993-02-08 1994-08-19 Olympus Optical Co Ltd Method and equipment for calibration for image measuring apparatus
JPH07281105A (en) * 1994-02-21 1995-10-27 Olympus Optical Co Ltd Endoscope device
CN106361281A (en) * 2016-08-31 2017-02-01 北京数字精准医疗科技有限公司 Fluorescent real-time imaging and fusing method and device
CN109730683A (en) * 2018-12-21 2019-05-10 重庆金山医疗器械有限公司 Endoscope object size calculation method and analysis system

Also Published As

Publication number Publication date
CN110811495A (en) 2020-02-21

Similar Documents

Publication Publication Date Title
CN110811495B (en) Focus size measuring method and system of fluorescence endoscope and computer equipment
US10706610B2 (en) Method for displaying an object
CN107874832B (en) Orthopedic surgical instrument navigation system and method
US9460536B2 (en) Endoscope system and method for operating endoscope system that display an organ model image to which an endoscopic image is pasted
ES2540953T3 (en) Device and method to determine a skin inflammation score
CN108291925B (en) Information processing device, speckle imaging system, and information processing method
JP2010531198A5 (en)
JP6478136B1 (en) Endoscope system and operation method of endoscope system
WO2017141524A1 (en) Imaging device, imaging method, and imaging system
CN107249427A (en) Medical treatment device, medical image generation method and medical image generation program
JP2018202104A (en) Endoscope system, and endoscopic image acquisition method
CN115887003A (en) Registration method and device of surgical navigation system and surgical navigation system
KR101374295B1 (en) Apparatus for ocular and method for measuring treatment position thereof
CN114980800B (en) Refractive pattern generation method, device and computer readable storage medium
US20210374981A1 (en) Apparatus and method for wide-field hyperspectral imaging
KR20210150695A (en) Image-based size estimation system and method for calculating lesion size through endoscopic imaging
CN113658243B (en) Fundus three-dimensional model establishing method, fundus camera apparatus, and storage medium
US9934569B2 (en) Image processing method and apparatus for tracking a site in a series of images
Field et al. Stereo endoscopy as a 3-D measurement tool
CN112740666A (en) System and method for multi-modal depth sensing in an automated surgical robotic vision system
US9820649B2 (en) Volume measuring device, volume measuring method, and volume measuring program for three-dimensional tomographic image
CN113936055A (en) Train brake shoe residual thickness measuring method, system, storage medium and computer equipment
JP2018128759A (en) Shininess removing device
CN115908121B (en) Endoscope registration method, device and calibration system
EP4019942A1 (en) Biological tissue identification method, biological tissue identification device, and biological tissue identification program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant