CN106959571B

CN106959571B - Multispectral projection and imaging device and multispectral projection method

Info

Publication number: CN106959571B
Application number: CN201710223871.3A
Authority: CN
Inventors: 黄忠守
Original assignee: Expantrum Optoelectronics
Current assignee: Expantrum Optoelectronics
Priority date: 2017-04-07
Filing date: 2017-04-07
Publication date: 2022-07-08
Anticipated expiration: 2037-04-07
Also published as: CN106959571A

Abstract

The invention provides a multispectral projection and camera device and a multispectral projection method. The multispectral projection device comprises: a first half mirror; the first half-mirror is used for reflecting the light emitted by the first half-mirror to the second half-mirror; the second light path emitted by the first light source is reflected to a space or an object through the first semi-transparent reflector, and the first video acquisition assembly acquires image information of a first type of spectral range of the space or the object in real time; and the processing unit is respectively connected with the projection element and the first video acquisition assembly, corrects the pre-stored images in the first spectral range and the associated images in other spectral ranges according to the real-time acquired image information in the first spectral range and projects the images to a space or an object. The multispectral projection and camera device and the multispectral projection method provided by the invention can accurately project image information in a plurality of spectral ranges to a space or an object.

Description

Multispectral projection and imaging device and multispectral projection method

Technical Field

The invention relates to the field of medical auxiliary equipment, in particular to a multispectral projection and camera device and a multispectral projection method.

Background

The structures and tissues inside the human body are not directly visible to the human eye. It is difficult to accurately find and locate subcutaneous internal structures and tissues relying solely on the external contour of the human body and knowledge of the anatomy of the human body.

Human blood vessels are hidden under the epidermis and are often shielded by subcutaneous fat and even bones, and visible light image signals reflected from human subcutaneous tissues in a visible light environment are weak and mixed with various noises and ghosts, and even are not visible to human eyes at all. Although a doctor often requires the patient to clench the fist or flap the skin of the puncture site to make the blood vessels more visible before puncturing, the visibility of the subcutaneous blood vessels is still not very desirable depending on the age of the patient, the thickness of the subcutaneous fat, and other factors. According to the invisible blood vessel image and medical knowledge, the puncture of the blood vessel is often misplaced, which causes the pain of the patient, delays the treatment time and even causes the injection accident. In addition to blood drawing and injection directly performed on blood vessels, acupuncture and other medical procedures require accurate knowledge of the location of blood vessels in order to avoid blood vessels or to perform special treatment on blood vessels during the procedure.

Disclosure of Invention

In view of the defects in the prior art, the present invention provides a multispectral projection device and method, a multispectral imaging device, a multispectral projection device and an imaging device, which overcome the difficulties in the prior art and can accurately project the position of a blood vessel so as to avoid the blood vessel or perform special treatment on the blood vessel during operation.

According to one aspect of the present invention, there is provided a multispectral projection device, comprising: a first half mirror; the first optical path of emergent light of the projection element penetrates through the first half-mirror; the second light path emitted by the first light source is reflected to the entity/virtual space or the entity/virtual object through the first half-mirror, and the first video acquisition component acquires the image information of the entity/virtual space or the entity/virtual object in real time in a first type of spectral range reflected by the first half-mirror from the second light path; a processing unit connected to the projection element and the first video capture assembly, respectively, the processing unit prestores image information of various spectral ranges of a solid/virtual space or a solid/virtual object, the image information of the multi-class spectral range comprises image information of a first class spectral range and image information of other classes of spectral ranges, the pre-stored image information of the first class spectral range and the pre-stored image information of the other classes of spectral ranges have image position mapping relations which are related to each other, according to the comparison between the image information of the first kind of spectral range acquired in real time and the image information of the pre-stored first kind of spectral range, the pre-stored image information of the first kind of spectral range and the associated image information of other kinds of spectral ranges are corrected and projected to the entity/virtual space or the entity/virtual object through the projection element.

Preferably, the projection element and the first video capture assembly are located on either side of the first half mirror.

Preferably, the first and second optical paths are substantially perpendicular to each other.

Preferably, an included angle between the first light path and the second light path is greater than or equal to 90 degrees.

Preferably, the first light source surrounds the first video capture assembly.

Preferably, the first light source is visible light and/or near infrared light. The visible light is herein referred to as light visible to the human eye and has a wavelength of from 400nm to 760 nm. The wavelength range of near infrared light is defined differently in the medical, industrial and military fields, typically from 760nm to around 3000 nm. Correspondingly, the first video acquisition assembly is one of an image sensor sensitive to only visible light, an image sensor sensitive to only near-infrared light, or a sensor sensitive to both visible light and near-infrared light and capable of selectively outputting one or more spectral images.

Preferably, the image information of the multiple types of spectral ranges includes a visible light picture, a near infrared light picture and an X-ray picture which are associated with each other at a point-to-point plane position.

Preferably, the image information of multiple spectral ranges is three-dimensional image information of multiple spectral ranges, which includes visible light three-dimensional image information, near infrared light three-dimensional image information, and X-ray three-dimensional image information associated with each other at point-to-point spatial positions.

Preferably, a rotating mechanism supporting the physical space or physical object is further included.

Preferably, the method further comprises the following steps: the second light source is positioned between the second light source and the first half-lens, and a third light path emitted by the second light source penetrates through the entity/virtual space or the entity/virtual object, wherein the first video acquisition component is also used for acquiring image information of a second type of spectral range of the entity/virtual space or the entity/virtual object reflected by the first half-lens from the third light path in real time; the processing unit is also used for correcting the related image information of other spectral ranges according to the image information of the second spectral range acquired in real time and projecting the image information to a solid/virtual space or a solid/virtual object through the projection element.

According to another aspect of the present invention, there is also provided a multispectral projection method using the above multispectral projection apparatus, the multispectral projection method including: the first video acquisition component acquires image information of a first type of spectral range of the second light path of the entity/virtual space or the entity/virtual object emitted from the first light source by the first half-lens reflector in real time; the processing unit corrects the associated image information of other spectral ranges according to the comparison between the image information of the first spectral range acquired in real time and the pre-stored image information of the first spectral range, wherein the processing unit pre-stores the image information of multiple spectral ranges of an actual space or an object, the image information of the multiple spectral ranges comprises the image information of the first spectral range and the image information of the other spectral ranges, and the pre-stored image information of the first spectral range and the image information of the other spectral ranges have image position mapping relations which are associated with each other; and a first light path of emergent light of the projection element penetrates through the first half-transmission reflector so as to project the corrected image information of other spectral ranges to an entity/virtual space or an entity/virtual object.

Preferably, the processing unit corrects the associated image information of other spectral ranges in real time according to the comparison between the image information of the first spectral range acquired in real time and the pre-stored image information of the first spectral range, and projects the corrected image information of other spectral ranges to the entity/virtual space or the entity/virtual object in real time.

Preferably, the image information of the multiple spectral ranges is three-dimensional image information of the multiple spectral ranges, and the processing unit corrects the associated image information of other spectral ranges according to a comparison between the image information of the first spectral range acquired in real time and the pre-stored image information of the first spectral range, and projects the image information of other spectral ranges to the entity/virtual space or the entity/virtual object through the projection element further includes: and processing the three-dimensional image information of the multiple spectral ranges into planar image information of the multiple spectral ranges correspondingly associated with the image information of the first spectral range acquired in real time, wherein the planar image information of the multiple spectral ranges retains part or all of the image information in the depth direction vertical to the planar image information of the multiple spectral ranges.

Preferably, the image information of the multiple spectral ranges includes multiple X-ray pictures, the multiple X-ray pictures respectively correspond to X-rays with different energies, and the processing unit selectively selects one or more of the multiple X-ray pictures to project to a solid/virtual space or a solid/virtual object.

According to another aspect of the present invention, there is also provided a multispectral imaging device, including: a second half mirror; a fourth light path emitted by the third light source is reflected to an entity/virtual space or an entity/virtual object through the second semi-transparent reflector, and the second video acquisition assembly acquires visible light image information and/or near infrared light image information of the entity/virtual space or the entity/virtual object reflected by the second semi-transparent reflector from the fourth light path in real time; the X-ray video acquisition assembly acquires image information of the X-ray light source penetrating through the entity/virtual space or the entity/virtual object and the second semi-transparent reflector along the fifth light path; and the processing unit is respectively connected with the X-ray video acquisition assembly and the video acquisition assembly.

Preferably, the X-ray video capture assembly and the second video capture assembly are respectively located on two sides of the second half-mirror.

Preferably, the third light source is visible light and/or near infrared light, and correspondingly, the image acquisition component is one of an image sensor sensitive to only visible light, an image sensor sensitive to only near infrared light, or a sensor sensitive to both visible light and near infrared light and capable of selectively outputting one or more spectral images.

Preferably, the processing unit stores the visible light picture and the near infrared light picture acquired by the second video acquisition component and the X-ray picture acquired by the X-ray video acquisition component, and establishes a point-to-point planar position mapping relationship for the visible light picture, the near infrared light picture and the X-ray picture.

Preferably, the processing unit stores visible light three-dimensional image information, near infrared light three-dimensional image information and X-ray three-dimensional image information acquired by the second video acquisition assembly, and establishes a point-to-point spatial position mapping relationship for the visible light three-dimensional image information, the near infrared light three-dimensional image information and the X-ray three-dimensional image information.

Preferably, the X-ray light source emits X-rays with different energies, the X-ray video capturing assembly captures a plurality of X-ray pictures corresponding to the X-rays with different energies, and the processing unit stores and stores the plurality of X-ray pictures.

According to another aspect of the present invention, there is also provided a multispectral projection and imaging apparatus, comprising: a first half mirror; the first half-mirror is used for reflecting the light emitted by the projection element to the second half-mirror; the second light path emitted by the first light source is reflected to an entity/virtual space or an entity/virtual object through the first half-mirror, and the first video acquisition component acquires image information of a first type of spectral range of the entity/virtual space or the entity/virtual object reflected by the first half-mirror from the second light path in real time; a second half mirror; a second video collecting component and a third light source, wherein a fourth light path emitted by the third light source is reflected to an entity/virtual space or an entity/virtual object through the second semi-transparent reflector, and the second video collecting component collects visible light image information and/or near infrared light image information of the entity/virtual space or the entity/virtual object reflected by the second semi-transparent reflector from the fourth light path in real time; the X-ray video acquisition assembly acquires image information of the X-ray light source penetrating through the entity/virtual space or the entity/virtual object and the second semi-transparent reflector along the fifth light path; the processing unit is respectively connected with the projection element, the first video acquisition component, the second video acquisition component and the X-ray video acquisition component, image information of multiple spectral ranges acquired by the second video acquisition component and the X-ray video acquisition component is prestored in a shooting stage, the image information of the multiple spectral ranges comprises image information of a first spectral range and image information of other spectral ranges, the image information of the first spectral range is acquired by the second video acquisition component, the image information of the other spectral ranges is at least acquired by the X-ray video acquisition component, and the prestored image information of the first spectral range and the prestored image information of the other spectral ranges have image position mapping relations which are related to each other; in the projection stage, according to the comparison between the image information of the first kind of spectral range acquired in real time and the pre-stored image information of the first kind of spectral range, the pre-stored image information of the first kind of spectral range and the associated image information of other kinds of spectral ranges are corrected and the entity/virtual space or the entity/virtual object is projected through the projection element.

In view of this, the multispectral projection device and method, the multispectral imaging device, the multispectral projection and the imaging device project relevant information of structures or tissues in a human body to the surface of the human body, so that the visibility of subcutaneous blood vessels and/or bones is greatly improved, human eyes can directly observe the surface of the human body in real time, the positions of the blood vessels and/or the bones and the relative positions of the blood vessels and the bones can be accurately projected through image correction when the surface of the human body is distorted, the blood vessels can be avoided or specially treated when the operation is convenient, and diagnosis and treatment of the internal structures and the tissues of the human body are facilitated.

Drawings

Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:

FIG. 1 is a schematic diagram of a multispectral projection apparatus according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of another multispectral projection device according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of another multispectral projection device according to an embodiment of the present invention;

FIG. 4 is a flow chart of a multi-spectral projection method according to an embodiment of the invention;

FIG. 5 is a schematic diagram of a multispectral imaging device according to an embodiment of the present invention;

FIG. 6 is a timing diagram of multi-spectral projection according to an embodiment of the present invention;

FIG. 7 is another timing diagram of multi-spectral projection according to an embodiment of the invention;

FIG. 8 is a diagram illustrating image information correction for multiple spectral ranges of multi-spectral projection, in accordance with an embodiment of the present invention;

FIG. 9 is a diagram illustrating image information correction for multiple spectral ranges of multi-spectral projection according to another embodiment of the present invention.

Reference numerals

1 object to be operated

11 rotating mechanism

17 multiple types of spectral range image information

2X-ray light source

21 fifth light path

3X-ray video acquisition assembly

4 third light source

41 first light source

42 second light source

43 third light path

5 second video acquisition component

51 fourth light path

52 electric wire

6 first video acquisition subassembly

61 second light path

62 electric wire

8 second half mirror

81 first half mirror

82 included angle

85 System

9 projection element

91 first light path

92 electric wire

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and thus their repetitive description will be omitted.

The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. In some instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring the invention.

In order to solve the defects of the prior art, the invention provides a multispectral projection device and method, a multispectral camera device, a multispectral projection device and a camera device. The physical space may represent an actual environment space, the virtual space may be a virtual environment space implemented by a three-dimensional projection technology such as holographic projection, the physical object may represent an object (e.g., an object such as a curtain or human skin) in the actual space, and the virtual object may be a virtual object implemented by a three-dimensional projection technology such as holographic projection (e.g., a virtual palm obtained by holographic projection). In the following, each embodiment will take a physical object as an operated object as an example to describe each apparatus and method provided by the present invention, it should be understood that the present invention is not limited thereto, and can also be applied to a physical/virtual space and a virtual object, and the operated object can be observed only without performing subsequent operations.

Referring first to fig. 1, fig. 1 is a schematic diagram of a multispectral projection device according to an embodiment of the present invention. The multispectral projection device comprises a first half-mirror 81, a projection element 9, a first video acquisition component 6, a first light source 41 and a processing unit 7.

The first optical path 91 of the exit light of the projection element 9 penetrates the first half mirror 81 to reach the surface of the object 1 to be operated. The projection element 9 and the first video acquisition assembly 6 may be located on either side of the first half mirror 81.

Optionally, a first light source 41 surrounds the first video capture assembly 6. The second optical path 61 emitted from the first light source 41 is reflected by the first half mirror 81 to the object 1 to be operated. The first video acquisition assembly 6 acquires image information of the first type of spectral range of the operated object 1 reflected by the first half mirror 81 from the second optical path 62 in real time. In other words, the second optical path 61 emitted from the first light source 41 is reflected by the first half mirror 81 to the operated object 1 (and after a certain depth below the surface of the operated object 1), reflected by the operated object 1 to reach the surface of the first half mirror 81, and then reflected to the first video capturing component 6. The first light path is preferably orthogonal to the second light path.

The first light source 41 is optionally visible light, correspondingly the first video capture assembly 6 is an image sensor sensitive only to visible light; the first light source 41 is optionally near infrared light, correspondingly the first video capture assembly 6 is an image sensor sensitive only to near infrared light; the first light source 41 is selectively visible and near infrared light, and correspondingly the first video acquisition assembly 6 is one of a sensor sensitive to both visible and near infrared light and capable of selectively outputting one or more spectral images. Correspondingly, the image information of the first kind of spectral range is a visible light picture and/or a near infrared light picture. Specifically, if the first video capturing assembly 6 is sensitive to both visible light and near infrared light, a device of a band pass filter or a reflective mirror is disposed in front of a lens or a photosensitive device of the first video capturing assembly 6, and the device selectively passes only visible light or only near infrared light, so that the first video capturing assembly 6 can selectively capture a visible light picture or a near infrared light picture of the operated object 1, so as to selectively sense visible light or near infrared light. Wherein the visible light wavelength range is about 400nm to 760 nm. The wavelength range of the near infrared light is about 760nm to 3000 nm.

The first half mirror 81 optionally includes a mirror and a coating disposed on a surface of the mirror. The coating enables the first light path 91 of the outgoing light of the projection element 9 to partially or substantially completely penetrate the mirror; and the second light path 61 of the first light source 41 can be partially or substantially totally reflected by the mirror. The coating is optionally provided on the side of the mirror facing the object 1 to be operated. For example, when the first video capture assembly 6 is sensitive to near infrared light, the first half mirror 81 is a so-called hot mirror that can transmit visible light and reflect near infrared light. The relative intensity of the transmitted light and the reflected light is related to the material and the thickness of the coating, the included angle between the semi-transparent reflector and the light path and the wavelength of the light. In order to adjust the relative intensities of the projected light and the reflected light, or to make the entire apparatus including the projection and image pickup device more compact, the angle between the semi-transparent mirror and the first optical path 91 may be greater or smaller than 45 degrees, that is, the angle between the first optical path 91 and the second optical path 61 may be greater or smaller than 90 degrees. In this embodiment, the angle between the first light path 91 and the second light path 61 is equal to 90 degrees.

The processing unit 7 is connected to the projection element 9 and the first video acquisition assembly 6, respectively. As shown in fig. 1, the processing unit 7 is connected to the projection element 9 via a wire 92 and to the first video capture assembly 6 via a wire 62. The invention is not limited thereto, and the connection of the processing unit 7 to the projection element 9 and the first video capturing assembly 6 may also be wireless. The processing unit 7 is optionally a computer, smart terminal or other electronic device with image processing capabilities. The processing unit 7 prestores image information 17 of a plurality of types of spectral ranges of the operated object 1. The image information 17 of the plurality of types of spectral ranges includes image information of a first type of spectral range and image information of other types of spectral ranges.

The pre-stored image information of the first kind of spectral range and the image information of other kind of spectral ranges have a correlated image position mapping relation. In some embodiments, the image information of multiple types of spectral ranges may include visible light pictures, near infrared light pictures, and X-ray pictures associated with each other at point-to-point plane positions. In other embodiments, the multi-type spectral range image information is multi-type spectral range three-dimensional image information, which includes visible light three-dimensional image information, near infrared light three-dimensional image information, and X-ray three-dimensional image information associated with each other in a point-to-point spatial position.

The processing unit 7 corrects the pre-stored image information of the first kind of spectral range and the associated image information of other kinds of spectral ranges according to the comparison between the image information of the first kind of spectral range acquired in real time and the pre-stored image information of the first kind of spectral range. The pre-stored image information of the first type of spectral range and the image information of the first type of spectral range collected in real time are image information of the same type of spectral range. For example, if the pre-stored image information in the first type of spectral range is a visible light picture, the first light source is a visible light source, and the first video collecting assembly 6 is sensitive to visible light to collect the visible light picture in real time; for another example, if the pre-stored image information in the first kind of spectral range is a near-infrared light picture, the first light source is a near-infrared light source, and the first video collecting assembly 6 is sensitive to the near-infrared light to collect the near-infrared light picture in real time; for another example, the pre-stored image information in the first kind of spectral range is a visible light picture and a near infrared light picture, the first light source can selectively emit visible light and near infrared light, and the first video capture assembly 6 is selectively sensitive to visible light and near infrared light to capture the visible light picture and the near infrared light picture in real time.

Specifically, the processing unit 7 compares the image information of the first kind of spectral range acquired in real time with the pre-stored image information of the first kind of spectral range point to point, and obtains a vector matrix which needs to correct the associated image information of other kinds of spectral ranges output to the projection element 9. The processing unit 7 transmits the corrected image information of the associated other spectral ranges to the projection element 9 through the electric wire 92, and projects the image information to the object 1 to be operated through the first half mirror 81 by the projection element 9.

In some specific embodiments, since the first video capturing assembly 6 captures the image information of the first kind of spectral range of the operated object 1 in real time, if the first video capturing assembly 6 captures the image information of the first kind of spectral range of the operated object 1 in real time, and if the projection element 9 projects the corrected image information of the associated other kinds of spectral ranges to the operated object 1, the image information of the first kind of spectral range of the operated object 1 captured in real time by the first video capturing assembly 6 also includes the image projected by the projection element 9.

In the present embodiment, the multispectral projection device further includes a rotating mechanism 11 that supports the object 1 to be operated. The rotating mechanism 11 is used to rotate the object 1 to be operated so that different sides of the object 1 to be operated can be projected. The rotating mechanism 11 may be a rotating table driven by a linear motor.

Fig. 2 is a schematic diagram of another multispectral projection apparatus according to an embodiment of the invention. The multispectral projection device shown in fig. 2 is similar to that shown in fig. 1, except that the first light path 91 and the second light path 61 are at an angle 82 greater than 90 degrees, as shown in fig. 1. When the included angle 82 between the first light path 91 and the second light path 61 is greater than or equal to 90 degrees, the overall system profile 85 can be made smaller than the embodiment shown in fig. 1, and the internal components can be more compact, thereby achieving miniaturization of the multispectral projection device.

Fig. 3 is a schematic diagram of another multispectral projection device according to an embodiment of the invention. The multispectral projection device shown in fig. 3 is similar in structure to that shown in fig. 1, except that in the embodiment shown in fig. 3, a second light source 42 is further included. The object 1 to be operated is positioned between the second light source 42 and the first half mirror 81, and the third light path 43 emitted from the second light source 42 passes through the object 1 to be operated. The first video collecting assembly 6 also collects the image information of the second type of spectral range of the operated object 1 reflected by the first half-mirror 81 from the third optical path 43 in real time. The processing unit 7 also corrects the associated image information of other spectral ranges of the type from the image information of the spectral range of the second type acquired in real time and projects it to the object 1 to be operated by means of the projection element 9. Optionally, the second light source 42 is an infrared light source, the first video capture assembly 6 is also sensitive to infrared light, and the image information in the second spectral range is an infrared light picture. In one embodiment, when the second light source 42 is mid-far infrared light and the object 1 to be operated is a palm, the second light source 42 is caused to illuminate the palm. Part of this infrared light will penetrate the palm of the person's hand to reach an infrared camera (e.g. the first video acquisition assembly 6) placed on the other side of the palm, so that the spatial position of the bones of the palm or the major arteries, veins can be obtained in real time, which information can be used to correct the image to be projected. The medium and far infrared light picture is different from the near infrared light picture, and blood vessels in deeper layers of human soft tissues can be patterned. Since infrared light is completely opaque to bone, the location of blood vessels can be correlated with the location of bone relatively accurately.

The multispectral projection method provided by the invention utilizes the multispectral projection device shown in fig. 1, and can refer to fig. 4, wherein fig. 4 totally shows three steps. The multispectral projection method will be described below in conjunction with the multispectral projection apparatus shown in fig. 1.

Step S301: the first video acquisition assembly 6 acquires the image information of the first type of spectral range of the operated object 1 in real time, wherein the second light path 61 emitted from the first light source 41 is reflected 81 by the first half-mirror.

Step S302: the processing unit 7 corrects the pre-stored image information of the first kind of spectral range and the associated image information of other kinds of spectral ranges according to the comparison between the image information of the first kind of spectral range acquired in real time and the pre-stored image information of the first kind of spectral range. Wherein, the processing unit 7 prestores the image information of multiple types of spectral ranges of the operated object 1. The image information of the multi-class spectral range comprises image information of a first-class spectral range and image information of other-class spectral ranges, and the pre-stored image information of the first-class spectral range and the pre-stored image information of the other-class spectral ranges have image position mapping relations which are related to each other. The processing unit 7 can correct the pre-stored image information of the first kind of spectral range and the associated image information of other kinds of spectral ranges in real time according to the comparison between the image information of the first kind of spectral range acquired in real time and the pre-stored image information of the first kind of spectral range. In particular, image processing and projection both occur in real time and at an extremely fast rate when image information for the spectral range of the first type is acquired in real time. Therefore, when the object 1 is shaken, the projected image continuously tracks the object 1, and the processed image is projected on the object 1.

Step S303: the first optical path 91 of the exit light of the projection element 9 penetrates the first half mirror 81 to project the corrected image information of the other spectral range to the object 1 to be operated.

For example, in a specific embodiment, the image information of the first kind of spectral range is a visible light picture, the image information of the other kind of spectral range includes an X-ray picture, and the operated object 1 is a palm. In order to project an X-ray picture onto the palm of the hand, it is first necessary to capture a visible light picture of the palm in real time using the first video capture assembly 6. The pre-stored visible light picture and the visible light picture collected in real time are compared with each other, and three-dimensional image processing is carried out, so that relative displacement and distortion of each part of the palm can be found when the visible light picture of the palm is collected in real time relative to the palm when the pre-stored visible light picture is collected. The displacement and distortion data are applied to the X-ray picture, and the exact positions of the current bones and blood vessels can be obtained by certain data operation. The processed image containing the current bone and blood vessels is projected with visible light onto the palm.

In another embodiment of the multispectral projection method, the image information of the plurality of types of spectral ranges includes image information of the plurality of types of spectral ranges of at least two sides of the object 1 to be manipulated, in consideration of the flip and distortion of the object 1 to be manipulated. When the object 1 to be operated is a palm, image information of various types of spectral ranges on both the front and back sides of the palm can be prestored. Thus, the blood vessel distribution on both sides of the skeleton and on both sides of the palm, especially the image of the blood vessel blocked by the skeleton, and the image information (such as visible light picture) of the first kind of spectral range on both sides of the palm can be completely obtained. Thus, when performing projection, if the palm is turned and twisted, the image data of other spectral ranges (such as X-ray pictures) on the reverse side are superimposed without any image missing, as determined by the processing unit 7.

In another embodiment of the multispectral projection method, the image information of a plurality of spectral ranges is three-dimensional image information of a plurality of spectral ranges, and the processing unit 7 corrects the associated image information of other spectral ranges according to a comparison between the image information of the first spectral range acquired in real time and the pre-stored image information of the first spectral range, and projects the image information of other spectral ranges to the operated object 1 through the projection element 9 further includes: and processing the three-dimensional image information of the multiple spectral ranges into planar image information of the multiple spectral ranges correspondingly associated with the image information of the first spectral range acquired in real time, wherein the planar image information of the multiple spectral ranges retains part or all of the image information in the depth direction vertical to the planar image information of the multiple spectral ranges.

Specifically, when the image information of multiple spectral ranges is the surface and internal information of the operated object 1 with a three-dimensional structure (for example, the image information of multiple spectral ranges shows three-dimensional X-ray pictures of the surface and internal blood vessels of the palm), the processing unit 7 can perform image processing to "flatten" the original three-dimensional image into a virtual two-dimensional picture, and then project the virtual two-dimensional picture on the surface of the operated object 1, such as the surface of the palm of a human. At this time, the blood vessel blocked behind the bone may be selectively not displayed, so as to show the anteroposterior position information of the blood vessel and the bone, that is, the compressed other dimension, or the partial information in the depth direction perpendicular to the planar image information of the plurality of spectral ranges. Or selectively displaying the information of all blood vessels and bones so as to display the related position information of all human tissues. In the same way, the lower blood vessels, which are occluded by the upper blood vessels, can also be selectively not displayed, so as to show the anteroposterior relationship of the blood vessels of different depths to each other.

Further, in some embodiments, if the object 1 is a human body, the skin, soft tissues such as fat and muscle of the human body are non-rigid materials and can be stretched and compressed. There is a certain error in determining the location of the bones and blood vessels under the skin, relying solely on the coordinates of the skin surface. To reduce or eliminate such errors, when the first video capture assembly 6 captures image information in the first spectral range in real time, the palm may be illuminated with a near-infrared light source that penetrates the skin to a certain depth, and the near-infrared light reflected from the palm may be captured by a corresponding near-infrared camera sensitive to the wavelength to obtain the position of the internal structure of the palm. The latest near infrared light is then used to correct the image to be projected again. In such an embodiment, the first video capture assembly 6 may be an image sensor sensitive to near infrared light, or an image sensor sensitive to both visible and near infrared spectral regions, with a filter to filter out near infrared or visible light before the imaging device.

In further embodiments, an appropriate wavelength of infrared light (e.g., mid-far infrared light) is selected to illuminate the object 1 (e.g., palm). Part of this infrared light will penetrate the palm of the person's hand to reach an infrared camera (e.g. the first video acquisition assembly 6) placed on the other side of the palm, so that the spatial position of the bones of the palm or the major arteries, veins can be obtained in real time. In this embodiment, the processing unit 7 can use this information to correct the image to be projected.

In further embodiments, the multiple types of spectral range image information includes multiple X-ray pictures. The plurality of X-ray pictures respectively correspond to X-rays with different energies. By inter-operation between a plurality of X-ray pictures), for example by subtraction), a clear X-ray picture of tissues of different densities can be obtained (e.g. an X-ray picture showing only soft body tissues, only bones, or only vascular systems). The processing unit 7 selectively selects one or more of the plurality of X-ray pictures to project to the operated object 1. For example, when the multispectral projection device of the present invention is used to project a visible image onto the surface of the object 1 (e.g., palm), it is possible to selectively project only soft tissues of the human body, or only bones, or only vascular systems.

In some specific applications, for example, when acupuncture therapy is performed, previous acupuncture therapists only judge the positions of acupuncture points of a patient according to the knowledge and experience of books and external structures of the body and external characteristics of the skin of the patient. And then performing acupuncture treatment according to the relatively rough judgment. Just as the proportional size of each person's body parts can vary widely, the exact location and depth of the acupuncture points under the skin of the person can also vary widely. According to the theory of modern traditional Chinese medicine, the acupuncture points of the human body and blood vessels, nerves and lymphatic vessels under the skin are strongly related. Using Magnetic Resonance Imaging (MRI), Diffusion Tensor Imaging (DTI), and angiography techniques, as well as far infrared imaging techniques (also known as thermography), one can also obtain sharp images of blood, nerves, and lymphatic vessels. The determination of the acupuncture points from blood vessels, nerve trunks and nerve endings, lymphatic vessels is much more accurate than the determination of acupuncture points from the external structures of the body and the skin appearance. The multispectral projection method provided by the invention can directly display the phenomenon that the acupuncture point map obtained by the imaging technologies is directly projected on the skin of a human body in a visible light mode. The acupuncture doctors can completely avoid worrying about the medical accidents caused by the reduced curative effect and even wrong position of the acupuncture caused by the deviation of the acupuncture from the acupuncture points.

Next, a multispectral imaging device according to an embodiment of the present invention will be described with reference to fig. 5. The multispectral camera device comprises a second transflector 8, a second video acquisition component 5, a third light source 4, an X-ray video acquisition component 3, an X-ray light source 2 and a processing unit 7.

A third light source 4 may surround the second video capture assembly 5. The fourth optical path 51 emitted from the third light source 4 is reflected by the second half mirror 8 to the object 1 to be operated. The second video collecting component 5 collects the visible light image information and/or the near infrared light image information of the operated object 1 reflected by the second semi-transparent reflector 8 from the fourth light path 51 in real time. In other words, the third optical path 51 emitted by the third light source 4 is reflected to the operated object 1 (and after a certain depth under the surface of the operated object 1) by the second half mirror 8, reflected by the operated object 1 to the surface of the second half mirror 8, and then reflected to the second video collecting assembly 5.

The third light source 4 is optionally visible light, correspondingly the second video capturing component 5 is an image sensor sensitive only to visible light; the third light source 4 is optionally near-infrared light, correspondingly, the second video capture component 5 is an image sensor sensitive only to near-infrared light; the third light source 4 is selectively visible and near infrared light, and correspondingly the second video capture assembly 5 is one of a sensor sensitive to both visible and near infrared light and capable of selectively outputting one or more spectral images. Specifically, if the second video capture assembly 5 is sensitive to both visible light and near-infrared light, a bandpass filter or a reflector is disposed in front of the lens or the photosensitive device of the second video capture assembly 5, and the bandpass filter or the reflector selectively passes only visible light or only near-infrared light, so that the second video capture assembly 5 can selectively capture a visible light picture or a near-infrared light picture of the operated object 1, so as to selectively sense visible light or near-infrared light. Wherein the visible light wavelength range is about 400nm to 760 nm. The wavelength range of the near infrared light is about 760nm to 3000 nm.

The second half-transparent reflector 8 is positioned between the X-ray video acquisition assembly 3 and the X-ray light source 2. The X-ray video acquisition assembly 3 acquires image information of the X-ray light source after penetrating through the operated object 1 and the second half-transparent reflector 8 along a fifth light path 21.

The X-ray video acquisition assembly 3 and the second video acquisition assembly 5 are respectively positioned at two sides of the second half-transparent reflector 8. The second partial reflector 8 optionally comprises a reflector and a coating provided on the surface of the reflector. The coating enables a fifth light path 21 of the emergent light of the X-ray light source 2 to penetrate the reflector; and the fourth light path 51 of the third light source 4 can be reflected by the mirror. The coating is optionally provided on the side of the mirror facing the object 1 to be operated.

The processing unit 7 is respectively connected with the X-ray video acquisition assembly 3 and the second video acquisition assembly 5. As shown in fig. 4, the processing unit 7 is connected to the X-ray video acquisition assembly 3 by a wire 32 and to the second video acquisition assembly 5 by a wire 52. The invention is not limited to this, and the connection between the processing unit 7 and the X-ray video capture assembly 3 and the second video capture assembly 5 may also be wireless. The processing unit 7 is optionally a computer, smart terminal or other electronic device with image processing capabilities. And the processing unit is respectively connected with the X-ray video acquisition assembly and the video acquisition assembly.

Preferably, the visible light image information and/or the near infrared light image information and the X-ray image information acquired by the second video acquisition assembly 5 and the X-ray video acquisition assembly 3 are stored in the processing unit 7 as image information of a first kind of spectral range (for example, a visible light picture and/or a near infrared light picture) and image information of other kind of spectral range (for example, an X-ray picture). And the processing unit 7 correlates the spatial coordinates of the points of the image information of the spectral range of the first type and the image information of the spectral range of the other type with each other, by which correlation the image information of the spectral range of the other type can be projected on the object 1 to be operated by means of some projection elements. For example, when the palm skin surface needs to be observed, image information (e.g., X-ray images, such as the positions of blood vessels and bones) of other spectral ranges that are already pre-associated can be projected on the skin surface with visible light at the same time. Although the fingers and the palm may be relatively displaced and even twisted, the relative positions of the tissues and bones below the skin and the epidermis may be kept stable to some extent.

In another preferred embodiment, the multispectral projection device as shown in any one of fig. 1 to 3 can be used to project image information in other spectral ranges. In such a preferred embodiment, the multispectral imaging device can be used to collect image information of multiple types of spectral ranges of at least two sides of the operated object 1, in consideration of the turning and distortion of the operated object 1 during the subsequent projection. When the object 1 is a palm, image information of various spectral ranges on both the front and back sides of the palm can be acquired. Thus, the blood vessel distribution on both sides of the skeleton and on both sides of the palm, especially the image of the blood vessel blocked by the skeleton, and the image information (such as visible light picture) of the first kind of spectral range on both sides of the palm can be completely obtained. Thus, when performing projection, if the palm is turned and twisted, the image data of other spectral ranges (such as X-ray pictures) on the reverse side are superimposed without any image missing, as determined by the processing unit 7.

In still another preferred example, the multispectral imaging device includes a rotating mechanism 11 that supports the object 1 to be operated. The rotating mechanism 11 can rotate the object 1 to be operated at the time of image acquisition. The rotating mechanism 11 may be a rotating table driven by a linear motor. Such a rotating mechanism 11 can be used not only for capturing front and back pictures of the object 1, but also for obtaining a three-dimensional image of the object 1. Correspondingly, the processing unit 7 stores the visible light three-dimensional image information, the near infrared light three-dimensional image information and the X-ray three-dimensional image information acquired by the second video acquisition assembly 5, and establishes a point-to-point spatial position mapping relationship for the visible light three-dimensional image information, the near infrared light three-dimensional image information and the X-ray three-dimensional image information.

In this way, it is possible to deal with deformation, twisting, turning, and the like of the object 1 to be operated (for example, palm). The X-ray three-dimensional image information of blood vessels, bones and other soft tissues in the palm can be obtained accurately and without loss by the X-ray video acquisition assembly 3, such as an X-ray computerized tomography scanner. While the second video capture assembly 5 captures all three-dimensional stereo pictures of visible and/or near-infrared light of the palm skin surface. Thus, the precise location of the underlying blood vessels, bones and other soft tissues can be calculated at any angle of the palm and projected onto the real skin.

Alternatively, during the phase of acquiring X-ray image information by the X-ray video acquisition assembly 3, images under different X-ray energy exposures may be acquired using a multi-X-ray energy imaging technique (e.g., DUAL-energy CT ANGIOGRAPHY technique DUAL-ENGERGY CT analog). By mutual operation between the images, for example, subtraction operation, clear images of tissues with different densities are obtained. The processing unit 7 can selectively select one or more of the plurality of X-ray pictures to project to the operated object 1 in the subsequent projection stage. For example, when the multispectral projection device of the present invention is used to project a visible image onto the surface of the object 1 (e.g., palm), it is possible to selectively project only soft tissues of the human body, or only bones, or only vascular systems.

FIG. 6 is a timing diagram of multi-spectral projection according to an embodiment of the present invention. Fig. 5 shows the lighting timing S1 of the first light source 41, the capturing timing I1 of the first video capturing assembly 6, and the projection timing P1 of the projection element 9. In other words, the projection element 9, the first video acquisition assembly 6, the processing unit 7, the projection element 9, and the time sequence and the process of re-projection. 41A is the first illumination of the first light source 41 corresponding to the first capture action 6A of the first video capture assembly 6 for which this interval is the first. The projection element 9 then attempts a first projection 9A, the projection phase 9A corresponding to the second acquisition action 6B of the first video acquisition assembly 6. The processing unit 7 performs image processing operations in the interval 7A according to the collected image information of the first-type spectral range and pre-stored image information of multiple types of spectral ranges of the processing unit 7. The corrected image information of the other spectral ranges is projected again onto the object 1 by using the visible light image. This projection corresponds to the zone 9B. Although it is sufficient to project image information of other spectral ranges that have been accurately positioned to the object 1 to be operated, then, as required, for example, the object 1 to be operated may shake slightly or even expand and contract during operation, and then the illumination 41B of the first light source 41 and the capturing action 6C of the first video capturing assembly 6 may be performed.

FIG. 7 is another timing diagram of multi-spectral projection according to an embodiment of the invention. Fig. 7 shows the lighting timing S2 of the first light source 41, the capturing timing I2 of the first video capturing assembly 6, and the projection timing P2 of the projection element 9. As shown in fig. 7, the capturing action of the first video capturing assembly 6, the projection of the projection element 9 and the image correction action of the processing unit 7 are performed during the capturing action of the first video capturing assembly 6, so as to correct the error of the projected image and track the shake and deformation of the operated object 1 in real time.

Referring to fig. 8 and 9, fig. 8 is a schematic diagram illustrating image information correction of multiple spectral ranges of multi-spectral projection according to an embodiment of the present invention, and fig. 9 is a schematic diagram illustrating image information correction of multiple spectral ranges of multi-spectral projection according to another embodiment of the present invention.

When the object 1 to be operated is a human palm, the observation angle and even the scale size thereof may be different from the originally taken visible light picture and X-ray picture or near-infrared light picture when actually observing the human body part. As shown in fig. 8, at this time, the parameters of the relative position and deformation of the human body part, or a vector matrix, can be obtained through the pre-stored visible light image 1A (pre-stored image information of the first kind of spectral range) and the visible light image 11A (pre-stored image information of the first kind of spectral range) acquired by the multi-spectral projection device in real time. The vector matrix is used for transforming an X-ray picture (such as 1B) or a near-infrared light picture (such as 1C) (image information of other spectral ranges) to be projected, and then the X-ray picture (such as 11B) and the near-infrared light picture (such as 11C) projected to the human body part are obtained. The coordinate transformation of the image can be used for the parallel movement and rotation of the human body part, and the equal proportion stretching or compression along one direction or two directions. In particular, FIG. 8 illustrates the process of multi-class spectral range image processing, rather than individual images. In the image capturing stage, the visible light picture 1A, X light picture 1B and the near infrared light picture 1C are obtained, and the plus sign in the figure indicates that the visible light picture 1A, X light picture 1B and the near infrared light picture 1C are used for processing to obtain the corresponding relation 1D between the visible light picture 1A, X light picture 1B and the near infrared light picture 1C (1D may also indicate the superposition of the visible light picture 1A, X light picture 1B and the near infrared light picture 1C). In the projection stage, the visible light picture 11A is obtained, the visible light picture 11A is compared with the visible light picture 1A, the X-ray picture 1B and the near-infrared light picture 1C are corrected, and then the X-ray picture 11B and the near-infrared light picture 11C are obtained, the visible light picture 11A, X can be projected by the superposition 11D of the light picture 11B and the near-infrared light picture 11C, or the X-ray picture 11B and the near-infrared light picture 11C can be projected separately.

In the same principle, when the human body part is stretched, compressed, and distorted in unequal proportion, the pre-stored X-ray picture or near-infrared light picture can be correspondingly transformed, so that each part of the human tissue is closer to the current actual position and size, as shown in fig. 9. In particular, FIG. 9 also illustrates the process of multi-class spectral range image processing, rather than individual images. In the image capturing stage, the visible light picture 1A, X light picture 1B and the near infrared light picture 1C are obtained, and the plus sign in the figure indicates that the visible light picture 1A, X light picture 1B and the near infrared light picture 1C are used for processing to obtain the corresponding relation 1D between the visible light picture 1A, X light picture 1B and the near infrared light picture 1C (1D may also indicate the superposition of the visible light picture 1A, X light picture 1B and the near infrared light picture 1C). In the projection stage, the visible light picture 12A is obtained, the visible light picture 12A is compared with the visible light picture 1A, the X-ray picture 12B and the near-infrared light picture 12C are corrected, and then the X-ray picture 12B and the near-infrared light picture 12C are obtained, the visible light picture 12A, X overlapping 12D of the light picture 12B and the near-infrared light picture 12C can be projected, or the X-ray picture 12B and the near-infrared light picture 12C can be projected separately.

According to yet another aspect of the present invention, there is also provided a multispectral projection and camera device incorporating the multispectral projection device shown in fig. 1 and the multispectral camera device shown in fig. 4. The multispectral projection and camera device comprises: a first half mirror; the first half-mirror is used for reflecting the light emitted by the first half-mirror to the second half-mirror; the second light path emitted by the first light source is reflected to the operated object through the first half-transparent reflector, and the first video acquisition component acquires image information of the operated object in a first spectral range reflected by the first half-transparent reflector from the second light path in real time; a second half mirror; the fourth light path emitted by the third light source is reflected to the operated object through the second semi-transparent reflector, and the second video acquisition component acquires visible light image information and/or near infrared light image information of the operated object from the fourth light path by the second semi-transparent reflector in real time; the X-ray video acquisition assembly acquires image information of the X-ray light source penetrating through the operated object and the second semi-transparent reflector along a fifth light path; the processing unit is respectively connected with the projection element, the first video acquisition component, the second video acquisition component and the X-ray video acquisition component, image information of a plurality of spectral ranges acquired by the second video acquisition component and the X-ray video acquisition component is prestored in a shooting stage, the image information of the plurality of spectral ranges comprises image information of a first spectral range and image information of other spectral ranges, the image information of the first spectral range is acquired by the second video acquisition component, the image information of the other spectral ranges is at least acquired by the X-ray video acquisition component, and the prestored image information of the first spectral range and the image information of the other spectral ranges have image position mapping relations which are mutually associated; in the projection stage, according to the comparison between the image information of the first kind of spectral range acquired in real time and the pre-stored image information of the first kind of spectral range, the pre-stored image information of the first kind of spectral range and the associated image information of other kinds of spectral ranges are corrected and projected through the projection element.

As shown above, in some embodiments, the present invention may implement the image capturing of a solid object and project images of various types of spectral ranges onto the same solid object or a virtual image (virtual object) of the same object; in other embodiments, the present invention may implement multispectral imaging in a physical space and project images of multiple spectral ranges corresponding to the same physical space or a virtual image (virtual space) in the same physical space; in still other embodiments, the present invention may also implement multispectral imaging of virtual objects and project images of corresponding multiple spectral ranges onto a physical object or a virtual image (virtual object) of the same object; in still other embodiments, the present invention may also enable multi-spectral imaging of a virtual space and project images of corresponding multi-spectral ranges onto the same physical space or onto a virtual image of the same physical space (virtual space). The present invention can implement the above-mentioned various embodiments, which are not described herein in detail.

In view of the above, the multispectral projection device and method, the multispectral camera device, the multispectral projection and camera device of the present invention project information related to structures or tissues inside a human body onto the surface of the human body, so that the visibility of subcutaneous blood vessels and/or bones is greatly improved, human eyes can directly observe the surface of the human body in real time, and the positions of the blood vessels and/or bones and the relative positions between the blood vessels and the bones can be accurately projected through image correction when the surface of the human body is distorted, so that the blood vessels can be avoided or specially treated during operation, and diagnosis and treatment of the internal structures and tissues of the human body are facilitated.

The basic concept and specific embodiments of the present invention have been described above. It should be noted that the present invention is not limited to the above-mentioned specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims without affecting the essence of the present invention. The invention is not limited to medical imaging applications described in the present invention for the purpose of illustrating the basic concepts, but of course also includes other fields of application such as industrial product and environment detection, personal identification, virtual space and augmented reality gaming, and business activities.

Claims

1. A multispectral projection device, comprising:

a first half mirror;

the first half-mirror is used for reflecting the light emitted by the projection element to the second half-mirror;

the second light path emitted by the first light source is reflected to an entity/virtual space or an entity/virtual object through the first half-mirror, and the first video acquisition component acquires image information of a first type of spectral range of the entity/virtual space or the entity/virtual object reflected by the first half-mirror from the second light path in real time;

a processing unit connected to the projection element and the first video capture assembly respectively, the processing unit prestores image information of various spectral ranges of the entity/virtual space or the entity/virtual object, the image information of the multi-class spectral range comprises image information of a first class spectral range and image information of other classes of spectral ranges, the pre-stored image information of the first class spectral range and the pre-stored image information of the other classes of spectral ranges have image position mapping relations which are related to each other, according to the comparison between the image information of the first kind of spectral range acquired in real time and the image information of the pre-stored first kind of spectral range, the pre-stored image of the first kind of spectral range and the associated image information of other kinds of spectral ranges are corrected and projected to the entity/virtual space or the entity/virtual object through the projection element.

2. The multi-spectral projection apparatus of claim 1 wherein said projection element and said first video capture assembly are located on opposite sides of said first half mirror.

3. The multispectral projection device of claim 1, wherein the first optical path and the second optical path are substantially perpendicular to each other.

4. The multi-spectral projection device of claim 1 wherein said first optical path is at an angle greater than or equal to 90 degrees to said second optical path.

5. The multi-spectral projection apparatus of claim 1 wherein said first light source surrounds said first video capture assembly.

6. The multi-spectral projection device of claim 1 wherein the first light source is visible and/or near infrared light and correspondingly the first video capture component is one of a visible light-only sensitive image sensor, a near infrared-only sensitive image sensor, or a sensor that is both visible and near infrared sensitive and can selectively output one or more spectral images.

7. The multispectral projection device of claim 1, wherein the image information for the plurality of spectral ranges includes a visible light picture, a near-infrared light picture, and an X-ray picture associated with each other at point-to-point planar locations.

8. The multispectral projection device of claim 1, wherein the multi-spectral-range image information is multi-spectral-range three-dimensional image information comprising visible light three-dimensional image information, near infrared light three-dimensional image information, and X-ray three-dimensional image information associated with each other at point-to-point spatial positions.

9. The multispectral projection device of claim 1, further comprising a rotating mechanism that supports the physical space or physical object.

10. The multispectral projection device of claim 1, further comprising:

a second light source, a solid/virtual space or a solid/virtual object, which is located between the second light source and the first half-mirror, and a third light path emitted from the second light source passes through the solid/virtual space or the solid/virtual object, wherein,

the first video acquisition component is also used for acquiring image information of a second type of spectral range of the entity/virtual space or the entity/virtual object reflected by the first half-mirror from the third light path in real time;

the processing unit is also used for correcting the related image information of other spectral ranges according to the image information of the second spectral range acquired in real time and projecting the image information to a solid/virtual space or a solid/virtual object through the projection element.

11. A multispectral projection method, using the multispectral projection apparatus as claimed in any one of claims 1 to 9, the multispectral projection method comprising:

the first video acquisition component acquires image information of a first type of spectral range of the second light path of the entity/virtual space or the entity/virtual object emitted from the first light source by the first half-lens reflector in real time;

the processing unit corrects the associated image information of other spectral ranges according to the comparison between the image information of the first spectral range acquired in real time and the pre-stored image information of the first spectral range, wherein the processing unit pre-stores the image information of a plurality of spectral ranges of the operated object, the image information of the plurality of spectral ranges comprises the image information of the first spectral range and the image information of other spectral ranges, and the pre-stored image information of the first spectral range and the image information of other spectral ranges have image position mapping relations which are associated with each other;

and a first light path of emergent light of the projection element penetrates through the first half-transmission reflector so as to project the corrected image information of other spectral ranges to an entity/virtual space or an entity/virtual object.

12. The multi-spectral projection method according to claim 11 wherein the processing unit corrects the pre-stored image of the first spectral range and the associated image information of the other spectral ranges in real time and projects the corrected image information of the other spectral ranges in real time to the physical/virtual space or the physical/virtual object based on a comparison of the image information of the first spectral range collected in real time and the pre-stored image information of the first spectral range.

13. The multi-spectral projection method according to claim 11, wherein the image information of multiple spectral ranges is three-dimensional image information of multiple spectral ranges, and the processing unit corrects the pre-stored image information of the first spectral range and the associated image information of other spectral ranges according to the comparison between the image information of the first spectral range collected in real time and the pre-stored image information of the first spectral range and projects the corrected image information of the first spectral range and the associated image information of other spectral ranges to the entity/virtual space or the entity/virtual object through the projection element further comprises:

and processing the three-dimensional image information of the multiple spectral ranges into planar image information of the multiple spectral ranges correspondingly associated with the image information of the first spectral range acquired in real time, wherein the planar image information of the multiple spectral ranges retains part or all of the image information in the depth direction vertical to the planar image information of the multiple spectral ranges.

14. The multi-spectral projection method of claim 11 wherein the image information for multiple spectral ranges includes multiple X-ray pictures, each of the multiple X-ray pictures corresponding to X-rays of different energy, the processing unit selectively selecting one or more of the multiple X-ray pictures to be projected onto the physical/virtual space or the physical/virtual object.

15. A multi-spectral projection and imaging apparatus, comprising:

a first half mirror;

a second half mirror;

a fourth light path emitted by the third light source is reflected to an entity/virtual space or an entity/virtual object through the second semi-transparent reflector, and the second video acquisition assembly acquires visible light image information and/or near infrared light image information of the entity/virtual space or the entity/virtual object reflected by the second semi-transparent reflector from the fourth light path in real time;

the X-ray video acquisition assembly acquires image information of the X-ray light source after penetrating through the entity/virtual space or the entity/virtual object and the second semi-transparent reflector along a fifth light path;

the processing unit is respectively connected with the projection element, the first video acquisition component, the second video acquisition component and the X-ray video acquisition component, image information of multiple spectral ranges acquired by the second video acquisition component and the X-ray video acquisition component is prestored in a shooting stage, the image information of the multiple spectral ranges comprises image information of a first spectral range and image information of other spectral ranges, the image information of the first spectral range is acquired by the second video acquisition component, the image information of the other spectral ranges is at least acquired by the X-ray video acquisition component, and the prestored image information of the first spectral range and the prestored image information of the other spectral ranges have image position mapping relations which are related to each other; in the projection stage, according to the comparison between the image information of the first kind of spectral range acquired in real time and the pre-stored image information of the first kind of spectral range, the pre-stored image information of the first kind of spectral range and the associated image information of other kinds of spectral ranges are corrected and projected to an entity/virtual space or an entity/virtual object through the projection element.