CN116965946A - Real-time projection system, projection device and projection method - Google Patents

Abstract

The invention provides a real-time projection system, a projection device and a projection method, wherein the real-time projection system comprises: the device comprises a light source assembly, a first image acquisition device, a second image acquisition device, a light blocking assembly, a projection device and a processor, wherein the light source assembly comprises a first laser generator, and emergent light of the first laser generator faces to an operation part; the first image acquisition device is used for acquiring a first image formed by light rays excited by the emergent light of the first laser generator; the second image acquisition device is used for acquiring visible light images of the operation part; the light blocking component intermittently blocks the second image acquisition device; the processor is used for generating a third image according to the first contour image and the visible light image of the first image, and controlling the projection device to project the first contour image in the third image when the light blocking assembly is in a blocking state. The invention provides a method for solving at least the technical problem that the accurate position of a focus is difficult to find in an operation.

Description

Real-time projection system, projection device and projection method

Technical Field

The present invention relates to the field of medical devices, and in particular, to a real-time projection system, a projection apparatus, and a projection method.

Background

Surgical navigation is a very useful surgical auxiliary technology, can be convenient to realize the correspondence of the image data before or during the operation of a patient and the actual anatomy structure of the patient, and in the operation process, a doctor is required to find the focus position, if the positioning is inaccurate, the operation failure is likely to be caused, and the combined application of the projection technology and the surgical navigation technology can effectively improve the operation precision by adding the image guidance in the surgical operation.

The existing projection system is applied to an operation, the position of a focus needs to be marked manually on an image of an organ before image projection, and the preoperative preparation time can be prolonged, so that the operation time is prolonged, the operation is not easy to complete rapidly, the accurate position of the focus is difficult to find by manual marking, and the operation effect is affected.

Therefore, a real-time projection system is urgently needed, which can be conveniently applied in combination with the surgical navigation technology to perform real-time projection, and achieve accurate registration of the virtual image and focus reality.

Disclosure of Invention

The invention provides a real-time projection system, a projection device and a projection method, which are used for at least solving the technical problem that the accurate position of a focus is difficult to find in an operation.

In order to achieve the above object, the present invention provides a real-time projection system comprising:

The light source assembly comprises a first laser generator, wherein emergent light of the first laser generator faces to the operation position, and the emergent light wavelength of the first laser generator is located outside a visible light wavelength range;

the first image acquisition device is used for acquiring a first image formed by light rays excited by the emergent light of the first laser generator;

a second image acquisition device for acquiring a visible light image of the surgical site, a field of view of the visible light image at least partially overlapping a field of view of the first image;

the light blocking assembly comprises a first light blocking unit, wherein the first light blocking unit intermittently blocks the second image acquisition device, so that the first light blocking unit is switched between a blocking state for blocking the second image acquisition device from acquiring the visible light image and an opening state for not blocking the second image acquisition device from acquiring the visible light image;

projection means for intermittently projecting onto the surgical site;

the first image acquisition device, the second image acquisition device and the projection device are electrically connected with the processor, and the processor is used for generating a third image according to a first contour image of the first image and the visible light image and controlling the projection device to project the first contour image in the third image when the first light blocking unit is in a blocking state.

The invention provides a real-time projection system, which is used for realizing the effect of real-time projection in a surgical navigation system, wherein a first image acquisition device and a second image acquisition device are respectively used for acquiring a first image and a visible light image, the first contour image obtained after the first image is processed by a processor and the visible light image are fused into a third image, and then the first contour image in the third image is projected into the view field of the first image by the projection device, so that the focus contour position of an organ or tissue to be subjected to surgery is displayed in real time.

Optionally, the first light blocking unit periodically blocks the second image acquisition device, and the period T of blocking the first light blocking unit is less than or equal to 1/24 seconds.

Optionally, the projection device includes a projection light source and a projection unit, the projection unit is configured to project the first contour image in the third image by using the projection light source, the projection light source is connected with the processor, and the projection light source intermittently emits projection illumination light, so that the projection unit intermittently projects the first contour image in the third image.

Optionally, the projection device further includes a second light blocking unit, where the second light blocking unit is disposed between the projection light source and the projection unit, and the second light blocking unit intermittently blocks the projection light source, so that the projection light source intermittently emits projection illumination light.

Optionally, the second light blocking unit includes a shutter.

Optionally, the system further comprises a ranging device, wherein the ranging device is in signal connection with the processor, the first image acquisition device, the projection device and the second image acquisition device.

Optionally, the display device further comprises an indication device and a diffraction element, wherein the indication device comprises an indication light source, and the indication light source irradiates in a part area irradiated by emergent light of the first laser generator to form a field of view of the first image; the diffraction element is disposed on an irradiation light path of the indication light source for irradiating the operation site.

Optionally, the first light blocking unit is a chopper or shutter.

Optionally, a path of light after excitation of the light emitted by the first laser generator, a path of light of the visible light image of the surgical site acquired by the second image acquisition device, and a path of light of the projection of the first contour image in the third image by the projection device are at least partially overlapped.

Optionally, a second laser generating device is also included, the second laser generating device being used to perform laser cutting operations.

The invention also provides a projection device, which is a projection device in the real-time projection system and comprises a projection light source and a projection unit, wherein the projection light source intermittently emits projection illumination light, and the projection unit is configured to project by using the projection light source.

The invention also provides a projection method based on the real-time projection system, which comprises the following steps:

emitting emergent light towards an operation position, and acquiring a first image formed by rays excited by the emergent light, wherein the emergent light is out of a visible light range;

intermittently acquiring visible light images of the surgical site, a field of view of the visible light images at least partially overlapping a field of view of the first image;

generating a first contour image according to the first image, and generating a third image according to the first contour image and the visible light image;

and in a blocking state of the first light blocking unit for blocking the second image acquisition device from acquiring the visible light image, projecting the first contour image in the third image to a field of view of the first image on the operation site.

Optionally, the intermittently acquiring the visible light image of the surgical site specifically includes intermittently shielding a second image acquiring device, so that the second image acquiring device intermittently acquires the visible light image, where the second image acquiring device is used for acquiring the visible light image of the surgical site.

Optionally, in a state that the first light blocking unit is in a blocking state of blocking the second image acquisition device from acquiring the visible light image, projecting the first contour image in the third image to a field of view of the first image on the surgical site, specifically including:

when the first light blocking unit is in a shielding state, controlling a projection light source of the projection device to be turned on;

and when the first light blocking unit is in an open state, controlling a projection light source of the projection device to be turned off.

According to the real-time projection system provided by the invention, the projection device projects the first contour image in the third image when the first light blocking unit is in the shielding state, so that the accuracy of the second image acquisition device for acquiring the visible light image is improved, the accuracy of the third image is improved, the image is stable under the vision of a doctor, the doctor can intuitively and clearly see the focus contour of an organ or tissue to be cut during operation, and the safety and the accuracy of operation are improved.

The real-time projection system provided by the invention can be conveniently applied to a surgical navigation system, is based on a time-sharing multiplexing mode, reduces the interference of image recognition, reduces fusion errors, improves positioning accuracy, thereby improving the projection effect of the real-time projection system, enabling medical intervention to be more effective, safer and more accurate, positioning the position of a focus accurately, enabling surgical operation to be safer and more accurate, and reducing doctor burden, thereby reducing surgical time and patient pain.

In addition to the technical problems, technical features constituting the technical solutions, and beneficial effects caused by the technical features of the technical solutions described above, other technical problems that can be solved by the real-time projection system provided by the embodiment of the present invention, other technical features included in the technical solutions, and beneficial effects caused by the technical features of the technical solutions are described in detail in the detailed description.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a real-time projection system according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a real-time projection system in a comparative example provided by the present invention;

fig. 3 is an example of a first image acquired by a first image acquisition device of a real-time projection system in a comparative example provided by the present invention;

FIG. 4 is an example of a first contour image derived from a first image for a real-time projection system in a comparative example provided by the present invention;

fig. 5 is an example of a visible light image acquired by the second image acquisition device of the real-time projection system in the comparative example provided by the present invention;

FIG. 6 is an example of a third image of a real-time projection system in a comparative example provided by the present invention;

FIG. 7 is an example one of an image of a third image of a real-time projection system in a comparative example provided by the present invention after a first contour image is projected;

FIG. 8 is an example two of an image of a third image of a real-time projection system of the comparative example provided by the present invention after a first contour image is projected;

FIG. 9 is an example three of an image in which a first contour image in a third image of a real-time projection system in a comparative example provided by the present invention is projected;

fig. 10 is an image example four after a first contour image in a third image of a real-time projection system in a comparative example provided by the present invention is projected;

FIG. 11 is an example of a first image acquired by a first image acquisition device of a real-time projection system provided by an embodiment of the present invention;

FIG. 12 is an example of a first contour image from a first image for a real-time projection system provided by an embodiment of the present invention;

fig. 13 is an example of a visible light image acquired by a second image acquisition device of a real-time projection system according to an embodiment of the present invention;

FIG. 14 is an example of a third image acquired by a real-time projection system provided by an embodiment of the present invention;

fig. 15 is an image example of a third image obtained by the real-time projection system according to the embodiment of the present invention after the first contour image is projected;

FIG. 16 is a graph showing the output power of a projection light source of a real-time projection system according to an embodiment of the present invention;

FIG. 17 is a graph showing the light transmittance of the first light blocking unit of the real-time projection system according to the embodiment of the present invention;

fig. 18 is a flowchart of a projection method of the real-time projection system according to an embodiment of the present invention.

Reference numerals illustrate:

10-a light source assembly;

11-a first laser generator;

20-first image acquisition means;

21-a first light splitting element;

22-an optical filter;

30-a second image acquisition device;

31-a second light splitting element;

A 40-light blocking assembly;

50-a projection device;

a 60-processor;

71-a distance measuring device;

72-indicating means;

73-a diffraction element;

81-a first image;

82-a first contour image;

83-visible light image;

84-a third image;

90-surgical site.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In order to improve the efficiency of the operation, before the operation, the focus needs to be manually marked before the image projection of the organ or the tissue, especially in the tumor treatment, the accuracy of matching the manually marked image with the actual position is low, and the doctor is inconvenient to find the position of the focus.

In view of the above background, the invention provides a real-time projection system, which can realize real-time projection of organs or tissues, so that medical intervention is more effective, safer and more accurate, and a doctor can intuitively and clearly see the focus outline of the organs or tissues to be subjected to surgery when performing the surgery, thereby improving the safety and the accuracy of the surgery operation and reducing the preparation time before the surgery.

Referring to FIG. 1, the present invention provides a real-time projection system comprising: the device comprises a light source assembly 10, a first image acquisition device 20, a second image acquisition device 30, a light blocking assembly 40, a projection device 50 and a processor 60, wherein the light source assembly 10 comprises a first laser generator 11, emergent light of the first laser generator 11 faces to a surgical site 90, and emergent light wavelength of the first laser generator 11 is outside a visible light wavelength range; referring to fig. 1 and 11, the first image acquiring device 20 is configured to acquire a first image 81 formed by light rays excited by the light emitted from the first laser generator 11; referring to fig. 1 and 13, the second image acquisition device 30 is configured to acquire a visible light image 83 of the surgical site 90, and a field of view of the visible light image 83 at least partially overlaps with a field of view of the first image 81; the light blocking assembly 40 includes a first light blocking unit that intermittently blocks the second image obtaining device 30 such that the first light blocking unit switches between a blocking state blocking the second image obtaining device 30 from obtaining the visible light image 83 and an open state not blocking the second image obtaining device 30 from obtaining the visible light image 83; projection device 50 is used to intermittently project onto the surgical site; the first image capturing device 20, the second image capturing device 30 and the projection device 50 are electrically connected to the processor 60, and the processor 60 is configured to generate a third image 84 according to the first contour image 82 of the first image 81 and the visible light image 83, and control the projection device 50 to project the first contour image 82 in the third image 84 when the first light blocking unit is in the blocking state.

The real-time projection system provided by the invention is used for realizing the effect of real-time projection in a surgical navigation system, the first image acquisition device 20 and the second image acquisition device 30 are respectively used for acquiring the first image 81 and the visible light image 83, the first contour image 82 obtained after the first image 81 is processed by the processor 60 and the visible light image 83 are fused into the third image 84, and the first contour image 82 in the third image 84 is projected into the view field of the first image 81 through the projection device 50, so that the focus contour position of an organ or tissue to be cut is displayed in real time, a doctor can intuitively and clearly see the focus contour of the organ or tissue to be cut when performing surgery, the safety and the accuracy of the surgery operation are improved, convenience is provided for smoothly performing the surgery, and the surgery efficiency is improved.

Referring to fig. 2, in some of the comparative examples provided real-time projection systems, there are: the light source assembly 10, the first image acquisition device 20, the second image acquisition device 30, the projection device 50 and the processor 60, wherein the light source assembly 10 includes a first laser generator 11, the emergent light of the first laser generator 11 faces the operation site 90, and the emergent light of the first laser generator 11 may be infrared light. The first image acquisition device 20 is configured to acquire, in real time, a first image 81 formed by light rays excited by the emitted light of the first laser generator 11; the second image real-time acquisition device 30 is configured to acquire a visible light image 83 of the surgical site 90, where a field of view of the visible light image 83 at least partially overlaps with a field of view of the first image 81; the processor 60 performs fusion processing on the first contour image 82 of the first image 81 and the visible light image 83 to generate a third image 84; the projection device 50 projects the third image 84 into the field of view of the first image 81 in real time, obtaining a field of view with the identified points.

Taking the operation site 90 as a liver for example, correspondingly, referring to fig. 3, which shows the first image 81 obtained by the real-time projection system provided by the comparative example, referring to fig. 4, which shows the first contour image 82 of the first image 81 obtained by the real-time projection system provided by the comparative example, referring to fig. 5, which shows the visible light image 83 obtained by the real-time projection system provided by the comparative example, referring to fig. 6, which shows an example of the third image 84 formed by fusing the first contour image 82 of the first image 81 obtained by the real-time projection system provided by the comparative example with the visible light image 83, since the third image 84 is fused in real time, referring to fig. 7, 8, 9 and 10, which show the third image 84 obtained at different times, respectively, it is apparent from fig. 7 to 10 that the fused images are in a continuously changing state.

With respect to this comparative example, the present invention provides a real-time projection system that avoids the first contour image 82 from being acquired by the second image acquisition device 30 when the first contour image 82 in the third image 84 is projected into the field of view of the first image 81, thereby avoiding affecting the accuracy of the visible light image 83 acquired by the second image acquisition device 30, and thus advantageously improving the accuracy of the fused third image 84. Therefore, in the present invention, the first light blocking unit intermittently blocks the second image acquisition device 30, so that the projection device 50 projects the first image in the first light blocking unit in the blocking state, which avoids the adverse effect of the projection of the first contour image 82 in the third image 84 on the acquisition of the visible light image 83 by the second image acquisition device 30, reduces the error of the fusion of the first contour image 82 of the first image 81 and the visible light image 83 into the third image 84, further in the operation process, the operation site 90 can be an organ or a tissue, and the like, and the focal position can be accurately positioned on the operation site 90, so that the doctor can intuitively and clearly see the focal contour to be cut during the operation, the operation is safer and more accurate, the burden of the doctor is reduced, and the operation time and the pain of the patient are reduced.

The real-time projection system provided by the invention reduces the interference of image recognition, reduces the fusion error and improves the positioning accuracy based on a time-sharing multiplexing mode, thereby improving the projection effect of the real-time projection system.

Alternatively, the emitted light from the first laser generator 11 is laser light, and irradiates the operation site 90 enriched with the fluorescent material, and the laser light excites the fluorescent material to emit light.

Alternatively, the first image acquisition device 20 may be a CCD camera or video camera for receiving infrared light reflected from the surgical site 90 excited by the first laser generator 11 to form a first image 81.

Alternatively, the processor 60 may be a computer or a computer program product, and the processor 60 receives the first image 81 and the visible light image 83 in real time.

Referring to fig. 11, an example of the first image 81 acquired by the first image acquisition device 20 is shown.

Referring to fig. 12, an example of a first contour image 82 of the first image 81 extracted from the first image 81 by the processor 60 using the feature extraction method is shown.

Referring to fig. 13, an example of the second image acquisition device 30 acquiring a visible light image 83 of the surgical site 90 is shown.

Referring to fig. 14, a third image 84 is formed by fusing a first contour image 82 of a first image 81 and a visible light image 83, and the third image 84 is fused in real time.

The processor 60 is configured to generate the third image 84 from the first contour image 82 and the visible light image 83 of the first image 81, and may be configured to extract the first contour image 82 of the first image 81 from the first image 81 by using a feature extraction method, as shown in fig. 11 and 12, and fuse the first contour image 82 of the first image 81 with the visible light image 83 in real time by using a built-in image processing module, thereby obtaining the third image 84.

It is easily understood that the first contour image 82 of the first image 81 is an edge contour of the first image 81.

In the case where the time division multiplexing method is not applied, that is, when the projection device 50 acquires the visible light image 83 of the operation site 90 by the second image acquisition device 30, the first contour image 82 is projected continuously in real time, and examples of the projected images obtained at any successive 1/24 th, 2/24 th, 3/24 th and 4/24 th seconds are shown with reference to fig. 7, 8, 9 and 10, respectively. In the case of not applying the time division multiplexing manner, since the projection of the first contour image 82 partially coincides with the shooting field of view of the visible light image 83, the projection of the first contour image 82 may cover a part of the visible light image 83, resulting in a problem that the acquired visible light image 83 is lost in information, so that the acquired visible light image 83 is inaccurate, and the accuracy of the fused third image 84 is further affected. As can be seen from fig. 7 to 10, the first contour image 82 in the third image 84 is changed in real time after being projected, which is easy to cause visual flicker, and cannot accurately show the fusion effect, so that the outline of the focus to be cut by the operation cannot be intuitively and clearly seen under the vision of a doctor, the safety of the operation is affected, and dizziness and discomfort are easily caused.

Referring to fig. 15, the effect of projecting the first contour image 82 in the third image 84 by controlling the projection device 50 in the shielding state of the first light blocking unit is shown in the present invention, so that the fusion effect can be continuously and stably displayed, the fusion precision is high, the image is stable under the vision of a doctor, and the doctor can intuitively and clearly see the contour of the focus to be cut.

Alternatively, referring to fig. 1, the second image acquisition device 30 may be a visible light camera that acquires a visible light image 83 of the surgical site 90 by photographing the surgical site 90. The field of view of the visible light image 83 captured by the second image capturing device 30 is at least coincident with the field of view of the first image 81 captured by the first image capturing device 20, and finally the obtained first image 81 corresponds to a local range or a whole range in the visible light image 83, so that when the real-time projection system provided by the invention is applied to a surgical navigation system, the visible light image 83 is a whole or partial image of a surgical site 90 including a focal part, the first image 81 is a partial image of the surgical site 90 including a focal part, and then the contour features of the first image 81 are extracted as a first contour image 82 of the first image 81, and after the first contour image 82 of the first image 81 and the visible light image 83 are fused as a third image 84, the contour of the focal to be surgically cut can be accurately positioned on the surgical site 90, thereby facilitating the operation of a doctor.

Optionally, the first light blocking unit periodically blocks the second image capturing device 30, where the period T of blocking of the first light blocking unit is less than or equal to 1/24 second. It is easy to understand that 1/24 second is generally considered as the longest time allowed for the human visual nervous system to move when each frame of picture is processed, and the period T blocked by the first light blocking unit is too large, which affects the smoothness of the visible light image 83 acquired by the second image acquisition device 30 in the process of implementing projection and causes the phenomenon of visual jamming.

Alternatively, the period T of the first light blocking unit blocking may be 1/24 second, 1/25 second, 1/30 second, 1/35 second, 1/40 second, etc.

Optionally, the light source assembly 10 further includes an illumination light source that irradiates toward the surgical site 90, the illumination range of the illumination light source coinciding with the field of view of the second image acquisition device 30. The illumination source may be a bulb or a light-emitting diode (LED) for providing visible light to the second image acquisition device 30 for acquiring a visible light image 83 of the surgical site 90.

The projection device 50 includes a projection light source and a projection unit, the projection unit being configured to project the first contour image 82 in the third image 84 using the projection light source, the projection light source being coupled to the processor 60, the projection light source intermittently emitting projection illumination light such that the projection unit intermittently projects the first contour image 82 in the third image 84. The processor 60 controls the projection light source to intermittently emit the projection illumination light, or the processor 60 controls the projection light source to intermittently turn on or off, so that the projection unit is in a projection working state when the projection light source is turned on, and is in a non-working state when the projection light source is turned off.

Alternatively, the projection unit is configured to project the first contour image 82 in the third image 84 using a projection light source, and may project the first contour image 82 in the third image 84 in the form of visible light.

Optionally, the projection light source periodically emits projection illumination light, the first light blocking unit periodically blocks the second image acquisition device 30, and when in use, the projection light source and the first light blocking unit form a cooperation in time, so that the processor 60 controls the projection light source to be turned on while the first light blocking unit blocks the second image acquisition device 30; or the first light blocking unit does not block the second image acquisition device 30, the processor 60 controls the projection light source to be turned off.

Fig. 16 is a graph showing output power of a projection light source of a real-time projection system according to an embodiment of the present invention, where an abscissa t represents time and an ordinate I represents light intensity. Fig. 17 is a graph of light transmittance of the first light blocking unit of the real-time projection system according to the embodiment of the present invention, wherein the abscissa t represents time and the ordinate τ represents light transmittance; as shown in fig. 16 and 17, in the period t1, the light transmittance of the first light blocking unit is 0, and the output power of the projection light source of the projection system is not 0; in the time period t2, the light transmittance of the first light blocking unit is 1, and the output power of the projection light source of the projection system is 0. The sum of the time period T1 and the time period T2 is equal to the period T of shielding of the first light blocking unit.

Optionally, the projection device 50 further includes a second light blocking unit disposed between the projection light source and the projection unit, where the second light blocking unit intermittently blocks the projection light source, so that the projection light source intermittently emits the projection illumination light. The second light blocking unit is arranged to intermittently block the projection light source also in order to cause the projection unit to intermittently project the first contour image 82 in the third image 84. The second light blocking unit and the first light blocking unit alternately work and form a cooperation in time, so that the projection light source and the second image acquisition device 30 realize time-sharing multiplexing in time.

Alternatively, the light wavelength of the projection light source is in the visible light range, for example, a laser having a wavelength between 490nm and 580nm may be used. The projection light source is a light source capable of periodically emitting light, for example, a pulsed laser.

Optionally, the projection unit comprises a digital micromirror device (Digital Micromirror Device, DMD for short) and a spatial light modulator (Spatial Light Modulator, SLM for short).

Optionally, the second light blocking unit comprises a shutter and/or a chopper.

Optionally, a distance measuring device is also included for measuring the distance between the first image acquisition device 20 and the surgical site 90, and for measuring the distance between the second image acquisition device 30 and the surgical site 90. It will be appreciated that only one manner of attachment of the distance measuring device is shown in fig. 1. The distance measuring device may be in signal connection with the processor, the first image capturing device 20, the projection device 50, the second image capturing device 30, the indication device 72, depending on the distance to be measured.

Alternatively, the distance measuring device may be an infrared distance meter. The processor calculates the time difference between the emission of the infrared ray and the reception of the reflected infrared ray based on the reflection of the infrared ray by the operation site 90, and calculates the imaging distance based on the time difference.

Optionally, an indication device is further included, and the indication device includes an indication light source, and the indication light source irradiates a partial area irradiated by the outgoing light of the first laser generator 11 to form a field of view of the first image 81. The indicating means is used to improve the accuracy of the fused third image 84.

Alternatively, the indication light source may be a light emitting diode, the purpose of which is to indicate the profile of the outgoing light emitted by the first laser generator 11.

Optionally, the present invention provides a real-time projection system, further comprising a filter 22, where the filter 22 is disposed on a light path of the first image capturing device 20 that receives the light reflected by the operation site 90, and the filter is used to select the required infrared light and filter out other light except the required infrared light.

Optionally, the present invention provides a real-time projection system, further comprising a diffraction element 73, the diffraction element 73 being disposed on an illumination light path that indicates that the light source is illuminating the surgical site. For example, the diffraction element 73 may be a fresnel lens.

Alternatively, the processor 60 calculates the imaging distance by the distance measuring device, adjusts the position of the diffraction element 73, and thereby obtains profile light of different diameters.

Optionally, the first light blocking unit is a chopper or shutter.

Optionally, the first light blocking unit may be an internal module of the second image capturing device 30, so that the second image capturing device 30 can intermittently capture the visible light image 83, and the first light blocking unit only needs to be capable of periodically blocking the visible light.

Optionally, the path of the light rays after excitation by the light emitted from the first laser generator 11, the path of the visible light image 83 of the surgical site 90 acquired by the second image acquisition device 30, and the path of the contour image of the third image 84 projected by the projection device 50 are at least partially overlapped. The object of this is to facilitate the fusion of the first contour image 82 of the first image 81 and the visible light image 83, to improve the fusion accuracy and to reduce the fusion time.

Optionally, the real-time projection system provided by the invention further includes a first light splitting element 21 and a second light splitting element 31, where the first light splitting element 21 and the second light splitting element 31 may adopt a transmission or reflection light splitting mode or a wavelength light splitting mode. The mirror image of the optical axis of the first image capturing device 20 in the second spectroscopic element 31 is on the same line as the optical axis of the second image capturing device 30, and the mirror image of the optical axis of the projection device 50 in the first spectroscopic element 21 is on the same line as the optical axis of the first image capturing device 20, that is, the mirror image of the optical axis of the projection device 50 in the first spectroscopic element 21 is also on the same line as the mirror image of the optical axis of the second image capturing device 30 in the second spectroscopic element 31, so that the common optical path is realized, so that the image planes of the first image 81, the visible light image 83 and the third image 84 projected by the projection device 50 are parallel, thereby easily realizing the fusion process, improving the fusion precision, and being capable of reducing the fusion time, thereby reducing the operation time.

It is to be understood that the positions of the first image capturing device 20, the second image capturing device 30 and the projection device 50 may be interchanged, so long as the first image capturing device 20, the second image capturing device 30 and the projection device 50 satisfy the condition of the common optical path, the solution that the common optical path may be satisfied by adding a beam splitter or interchanging the positional relationship of the first image capturing device 20, the second image capturing device 30 and the projection device 50 falls within the scope of the present application.

In the operation, since the first image capturing device 20, the second image capturing device 30 and the projection device 50 capture images in real time to perform projection, flickering of continuously changing the contour and position will occur continuously from the viewpoint of a doctor, and visual fatigue is likely to occur. The real-time projection system provided by the application can effectively avoid the condition of visual flicker and improve the visual experience of doctors.

Optionally, a second laser generating device is also included, and the second laser generating device is used for performing laser cutting operation. The second laser generating device may be a high-power laser generator, instead of a conventional scalpel, and performs operations such as incision on a focus of an organ or tissue at a position to be cut according to the calculated contour position of the first contour image 82 of the first image 81, so as to simplify the operations and reduce the risk of surgery.

The projection unit is connected to the processor 60 to perform real-time signal transmission, and the first image acquisition device 20 and the second image acquisition device 30 are connected to the processor 60 to perform real-time signal transmission, and the serial communication interface of the distance measuring device 71 is connected to the processor 60 to perform real-time signal transmission.

It is easy to understand that, when the first image capturing device 20 captures the first image 81, the second image capturing device 30 captures the visible light image 83 simultaneously, and at the same time, the projection device 50 projects the first contour image 82 in the third image 84 in a state that the first light blocking unit is in a blocking state that blocks the second image capturing device 30 from capturing the visible light image 83, so that the effect of realizing projection is visually presented, and a doctor can intuitively and clearly see the contour of the lesion of the portion to be resected when performing an operation.

According to the real-time projection system provided by the invention, the interference of image recognition is reduced, the fusion error is reduced and the positioning accuracy is improved by a time-sharing multiplexing mode, so that the projection effect of the operation navigation system is further improved.

The present invention also provides a projection device 50, which is the projection device 50 in the real-time projection system, and includes a projection light source and a projection unit, wherein the projection light source intermittently emits projection illumination light, and the projection unit is configured to project by using the projection light source. The projection unit is in a projection working state when the projection light source is turned on, and is in a non-projection non-working state when the projection light source is turned off.

The projection unit includes a digital micro-mirror device (Digital Micromirror Device, DMD for short) for implementing digital optical processing and a spatial light modulator (Spatial Light Modulator, SLM for short) that projects light from the projection light source to the surgical site 90 to project an image.

As for realizing that the projection light source intermittently emits projection illumination light, there are three different implementations:

firstly, the projection light source can be pulse light; the projection light source is intermittently started and stopped, and can be periodically started and stopped; and thirdly, a second light blocking unit is arranged between the projection light source and the projection unit, the second light blocking unit can block the projection light source intermittently, and the projection light source can be blocked by the second light blocking unit periodically, so that the projection light source can emit projection illumination light intermittently, and the projection unit can realize intermittent projection.

The projection device 50 provided by the invention can realize intermittent projection, and when being used for the projection device 50 in the real-time projection system, the projection device 50 is matched with the second image acquisition device 30 to improve visual effect. The period of time in which the first light blocking unit blocks the second image acquisition device 30 is exactly the period of time in which the projection light source emits the projection illumination light, and the projection unit projects in the period of time, and the second image acquisition device 30 cannot acquire the visible light image 83; the time period in which the first light blocking unit does not block the second image acquisition device 30 is exactly the time period in which the projection light source does not emit projection illumination light, and the second image acquisition device 30 acquires the visible light image 83 in the time period, and the projection unit does not project, so that the above-mentioned real-time projection system is facilitated to realize time division multiplexing.

Referring to fig. 1 and 18, the present invention further provides a projection method, based on the above real-time projection system, the projection method includes:

s10, emitting emergent light towards the operation position, and acquiring a first image formed by rays excited by the emergent light, wherein the emergent light is out of the visible light range.

In step S10, the emitted light from the first laser generator 11 is laser light, and irradiates the operation site 90 enriched with the fluorescent material to excite the fluorescent material to emit light. A first image 81 is formed by acquiring an image of the surgical site 90 under excitation of the emitted light by the first image acquisition device 20.

S20, intermittently acquiring visible light images of the operation part, wherein the field of view of the visible light images at least partially overlaps with the field of view of the first image.

In the step S20, the visible light image 83 of the operation site 90 is acquired by the second image acquisition device 30, wherein the intermittent acquisition of the visible light image 83 of the operation site 90 may be periodically shielding the second image acquisition device 30 by a first light blocking unit, which may be a chopper or a shutter.

S30, generating a first contour image according to the first image, and generating a third image according to the first contour image and the visible light image.

In step S30, after the first image 81 is processed by the processor 60, the contour of the first image 81 may be extracted from the first image 81, that is, the first contour image 82 of the first image 81 may be generated, and the obtained first contour image 82 of the first image 81 may be fused with the visible light image 83 by the processor 60 to form the third image 84. The processor 60 may be a computer or a computer program product.

S40, under the condition that the first light blocking unit is in a blocking state for blocking the second image acquisition device from acquiring the visible light image, the first contour image in the third image is projected to the view field of the first image on the operation part.

In step S40, the first contour image 82 in the third image 84 may be projected by the projection device 50 into the field of view of the first image 81. Since the first light blocking unit is in the blocking state, the second image acquisition device 30 cannot acquire the visible light image 83, and the projection device 50 projects the first contour image 82 of the third image 84 into the field of view of the first image 81 on the operation site 90.

The projection device 50 may intermittently project the third image 84 by the following three methods, the first being: the projection light source may be a pulsed laser; the second is that the projection light source is periodically turned on and off; the third is that a second light blocking unit is arranged between the projection light source and the projection unit, and the second light blocking unit periodically blocks the projection light source to realize that the projection light source periodically emits projection illumination light, so that the projection unit can realize periodic projection. In time, since the first light blocking unit periodically blocks the second image acquisition device, the projection period of the projection unit and the period of time for the second image acquisition device to acquire the visible light image are staggered and do not overlap.

The projection method provided by the invention is based on the real-time projection system, and can realize the effect of real-time projection, and the first contour image 82 in the third image 84 is projected to the field of view of the first image 81 on the operation position 90 under the condition that the first light blocking unit is in a shielding state, so that the interference on the identification of the visible light image 83 can be reduced, the fusion error of the third image 84 is reduced, the positioning precision of the first contour image 82 in the third image 84 during projection is improved, a doctor can intuitively and clearly see the focus contour of an organ or tissue to be operated during operation, and the safety and the accuracy of operation are improved.

Since the operation site 90 is always illuminated by the illumination light source when the projection is performed, the steps S10 and S20 are performed in substantially the same time period. In step S10, the emission of the outgoing light may be continuous. When the outgoing light in step S10 is continuously outgoing, the acquisition time of the visible light image 83 in step S20 is within the period of time when the outgoing light is continuously outgoing.

Optionally, in step S20, intermittently acquiring the visible light image of the surgical site, specifically includes intermittently shielding the second image acquiring device 30, so that the second image acquiring device 30 intermittently acquires the visible light image 83, where the second image acquiring device 30 is configured to acquire the visible light image 83 of the surgical site 90.

Alternatively, the second image capturing device 30 may be periodically blocked, so that the light obtained by the second image capturing device 30 changes periodically.

Optionally, referring to fig. 1 and 18, in step S40, in a state that the first light blocking unit is in a blocking state that blocks the second image capturing device from capturing the visible light image, the first contour image 82 in the third image is projected to the field of view of the first image on the surgical site, specifically including:

when the first light blocking unit is in a blocking state, controlling a projection light source of the projection device 50 to be turned on;

when the first light blocking unit is in the lifting state, the projection light source of the projection device 50 is controlled to be turned off; wherein the above actions are alternately repeated, thereby realizing the continuous effect of real-time projection.

It is easy to understand that the second image acquisition device 30 cannot acquire the visible light image 83 when the first light blocking unit is in the blocking state; the second image acquisition device 30 acquires the visible light image 83 with the first light blocking unit in the lifted state.

It should be noted that, the numerical values and numerical ranges referred to in the present application are approximate values, and may have a certain range of errors due to the influence of the manufacturing process, and those errors may be considered to be negligible by those skilled in the art.

In the description of the present invention, it should be understood that the terms "center", "length", "width", "thickness", "top", "bottom", "upper", "lower", "left", "right", "front", "rear", "vertical", "horizontal", "inner", "outer", "axial", "circumferential", etc. are used to indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, merely to facilitate description of the present invention and to simplify the description, and do not indicate or imply that the referred location or element must have a specific orientation, in a specific configuration and operation, and therefore should not be construed as limiting the present invention.

Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.

In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly and may be, for example, fixedly attached, detachably attached, or integrally formed; can be mechanically connected, electrically connected or can be communicated with each other; can be directly connected or indirectly connected through an intermediate medium, and can lead the interior of two elements to be communicated or lead the two elements to be in interaction relationship. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.

In the present invention, unless expressly stated or limited otherwise, a first feature "above" or "below" a second feature may include both the first and second features being in direct contact, as well as the first and second features not being in direct contact but being in contact with each other through additional features therebetween. Moreover, a first feature being "above," "over" and "on" a second feature includes the first feature being directly above and obliquely above the second feature, or simply indicating that the first feature is higher in level than the second feature. The first feature being "under", "below" and "beneath" the second feature includes the first feature being directly under and obliquely below the second feature, or simply means that the first feature is less level than the second feature.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.

Claims (14)

1. A real-time projection system, comprising:

the light source assembly comprises a first laser generator, wherein emergent light of the first laser generator faces to the operation position, and the emergent light wavelength of the first laser generator is located outside a visible light wavelength range;

the first image acquisition device is used for acquiring a first image formed by light rays excited by the emergent light of the first laser generator;

a second image acquisition device for acquiring a visible light image of the surgical site, a field of view of the visible light image at least partially overlapping a field of view of the first image;

the light blocking assembly comprises a first light blocking unit, wherein the first light blocking unit intermittently blocks the second image acquisition device, so that the first light blocking unit is switched between a blocking state for blocking the second image acquisition device from acquiring the visible light image and an opening state for not blocking the second image acquisition device from acquiring the visible light image;

projection means for intermittently projecting onto the surgical site;

The first image acquisition device, the second image acquisition device and the projection device are electrically connected with the processor, and the processor is used for generating a third image according to a first contour image of the first image and the visible light image and controlling the projection device to project the first contour image in the third image when the first light blocking unit is in a blocking state.

2. The real-time projection system of claim 1, wherein the first light blocking unit periodically blocks the second image acquisition device, and wherein a period T of the first light blocking unit blocking is less than or equal to 1/24 second.

3. The real-time projection system of claim 1, wherein the projection device comprises a projection light source and a projection unit, the projection unit configured to project the first contour image in the third image using the projection light source, the projection light source being coupled to the processor, the projection light source intermittently emitting projection illumination light to cause the projection unit to intermittently project the first contour image in the third image.

4. The real-time projection system of claim 3, wherein the projection device further comprises a second light blocking unit disposed between the projection light source and the projection unit, the second light blocking unit intermittently blocking the projection light source such that the projection light source intermittently emits projection illumination light.

5. The real-time projection system of claim 4, wherein the second light blocking unit comprises a shutter.

6. The real-time projection system of any of claims 1-5, further comprising a ranging device in signal communication with each of the processor, the first image acquisition device, the projection device, and the second image acquisition device.

7. The real-time projection system of any of claims 1-5, further comprising an indication device and a diffraction element, the indication device comprising an indication light source, the indication light source illuminating within a portion of the area illuminated by the outgoing light of the first laser generator, forming a field of view of the first image; the diffraction element is disposed on an irradiation light path of the indication light source for irradiating the operation site.

8. The real-time projection system of any of claims 1-5, wherein the first light blocking unit is a chopper or shutter.

9. The real-time projection system of any of claims 1-5, wherein a path of light rays after excitation by the light emitted from the first laser generator, a path of light rays for the second image acquisition device to acquire a visible light image of the surgical site, and a path of light rays for the projection device to project the first contour image in the third image at least partially coincide.

10. The real-time projection system of any of claims 1-5, further comprising a second laser generating device for performing a laser cutting procedure.

11. A projection device, characterized in that it is a projection device in a real-time projection system according to any one of claims 1-10, comprising a projection light source that intermittently emits projection illumination light, and a projection unit configured to project with the projection light source.

12. A projection method, characterized in that it is based on the real-time projection system according to any of claims 1-10, said projection method comprising:

emitting emergent light towards an operation position, and acquiring a first image formed by rays excited by the emergent light, wherein the emergent light is out of a visible light range;

intermittently acquiring visible light images of the surgical site, a field of view of the visible light images at least partially overlapping a field of view of the first image;

generating a first contour image according to the first image, and generating a third image according to the first contour image and the visible light image;

and in a blocking state of the first light blocking unit for blocking the second image acquisition device from acquiring the visible light image, projecting the first contour image in the third image to a field of view of the first image on the operation site.

13. The projection method according to claim 12, wherein the intermittently acquiring the visible light image of the surgical site, in particular, comprises intermittently shielding a second image acquisition device for intermittently acquiring the visible light image of the surgical site.

14. The projection method according to claim 12, wherein the projecting the first contour image in the third image to the field of view of the first image on the surgical site in a state where the first light blocking unit is in a blocking state blocking the second image acquisition device from acquiring the visible light image, specifically comprises:

when the first light blocking unit is in a shielding state, controlling a projection light source of the projection device to be turned on;

and when the first light blocking unit is in an open state, controlling a projection light source of the projection device to be turned off.