WO2016029403A1

WO2016029403A1 - Head-mounted molecular image navigation system

Info

Publication number: WO2016029403A1
Application number: PCT/CN2014/085396
Authority: WO
Inventors: 田捷; 迟崇魏; 杨鑫
Original assignee: 中国科学院自动化研究所
Priority date: 2014-08-28
Filing date: 2014-08-28
Publication date: 2016-03-03

Abstract

A head-mounted molecular image navigation system, comprising: a multi-spectral light source module for emitting visible light and near-infrared light to a detection region, a signal acquisition module for acquiring a near-infrared fluorescence image and a visible light image of an imaging object, a head-mounted system supporting module for bearing the multi-spectral light source module and the signal acquisition module so as to adjust illumination of the multi-spectral light source module to the detection region, and an image processing module for fusing the acquired near-infrared light image and the acquired visible light image, and outputting a fused image. According to the embodiments of the present invention, flexible usage of the device in applications of an image system is effectively achieved, and an application space of optical molecular image navigation is expanded.

Description

Head-mounted molecular imaging navigation system

Technical field

The invention relates to an imaging system, in particular to a head mounted molecular image navigation system. Background technique

As a new method and means of non-invasive visualization imaging technology, molecular imaging essentially reflects changes in the level of physiological molecules and changes in overall function of organisms caused by changes in molecular regulation. Therefore, studying the life activities of genes, biological macromolecules and cells in vivo at the molecular level is an important technology, including in vivo bio-optics based on molecular techniques, tomographic techniques, optical imaging techniques, and simulation methods. Basic research on imaging technology has become one of the hotspots and difficulties in the field of molecular imaging.

Molecular imaging equipment combines traditional medical imaging technology with modern molecular biology to observe physiological or pathological changes at the cellular or molecular level. It has the advantages of non-invasive, real-time, in vivo, high specificity, high sensitivity and high resolution imaging. . The use of molecular imaging technology can greatly speed up the development of drugs, shorten the pre-clinical research time of drugs, provide more accurate diagnosis, make the treatment plan best match the patient's genetic map, and help pharmaceutical companies develop personalized treatment. On the other hand, it can be applied in the field of biomedicine to achieve the objectives of quantitative analysis, image navigation and molecular typing in vivo. However, the system using this method is relatively complicated, and the ease of operation and the comfort of use need to be further improved.

Therefore, the present invention proposes a head-mounted molecular image navigation system for detecting an in-vivo target in a molecular image by a multi-spectral excitation method, and enhancing the application range of the application. Summary of the invention

The invention provides a head-mounted molecular image navigation system, comprising:

a multi-spectral light source module for illuminating the detection area with visible light and near-infrared light;

a signal acquisition module, configured to acquire a near-infrared fluorescence image and a visible light image of the imaged object; a head mounted system support module, configured to carry the multi-spectral light source module and the signal acquisition module, to adjust the multi-spectral light source module pair Irradiation of the detection area;

The image processing module is configured to perform image fusion on the collected near-infrared light image and the visible light image, and output the fused image. Embodiments of the present invention have the following technical effects:

1. Molecular image navigation and molecular imaging through head-on method, which improves the convenience while realizing the function.

2. The projection imaging method can guide the operator to pre-judge the imaging range, thus increasing the function of human-computer interaction.

3. The function of speech recognition can facilitate the operator to further liberate both hands in the process of using the system, thereby more accurately controlling the head-mounted molecular image navigation system.

4. Due to the feature value extraction method using threshold decomposition, the letter-to-back ratio is significantly improved, which helps the operator to guide the real-time precise operation according to the image. DRAWINGS

1 is a schematic structural view of a head mounted system supporting module according to an embodiment of the present invention;

2 is a block diagram of a head mounted molecular image navigation system in accordance with an embodiment of the present invention;

3 is a flow chart of an image processing method of a head-mounted molecular image navigation system in accordance with an embodiment of the present invention. detailed description

The present invention will be further described in detail below with reference to the specific embodiments of the invention.

Embodiments of the present invention provide a head-mounted molecular imaging navigation system based on excitation fluorescence imaging in molecular imaging. 1 is a schematic structural view of a head mounted system supporting module according to an embodiment of the present invention. 2 is a block diagram of a head mounted molecular image navigation system in accordance with an embodiment of the present invention. As shown in FIG. 2, the head-mounted molecular image navigation system may include a multi-spectral light source module 110 for providing light of a plurality of different spectral segments to illuminate a subject; and an optical signal acquisition module 120 for real-time acquisition and reception. The fluorescent excitation image and the visible light image of the inspection object; the head mounted system support module 130 is configured to adjust the comfort of the operator when wearing, and ensure the safe and effective imaging; the image processing module 140 is configured to perform image segmentation and feature extraction. , Processing such as image registration, fusion of visible light image and fluorescent image and output of fused image.

Next, the operations of the multi-spectral light source module 110, the optical signal acquisition module 120, the head mounted system support module 130, and the image processing module 140 will be described in detail. Multi-spectrum light source module 110 may include a cold light source 111, a near-infrared laser light source 112 and the coupler ₁₁₃₀ to a cold light source 111 for emitting visible objects subject. The cold light source 111 may be placed with a first band pass filter to transmit visible light having a wavelength of 400-650 nm. The near-infrared laser 113 is configured to emit near-infrared light having a center wavelength of, for example, 785 nm. The excitation light source can be extracted through the optical fiber. It is known to those skilled in the art that the embodiments of the present invention are not limited to the above implementations, and visible light and near-infrared light may also be emitted by other means known in the art. When the detection region is excited, based on the spectral separation method, the light source 111 and the near-infrared laser 112 are simultaneously emitted by a single optical fiber. Specifically, the light emitted from the visible light source and the near-infrared light source is coupled at the light exit port. A light source coupler 113 is provided at the coupling. The light source coupler 113 can be a diverging lens, and the light source is changed from a linear point source to a cone beam, which can enlarge the illumination area to achieve uniform illumination of the detection area by the excitation source. For example, an optical lens can be disposed at the light exit of the near-infrared laser 112, and the optical lens is inversely coupled with the output end of the laser to achieve an output of a large divergence angle of the light source. A mechanical fixing method may be used to fix one end of the optical fiber to the optical lens and the other end of the optical fiber to the head mounted system support module 130.

The optical signal acquisition module 120 can include a camera 121, a lens 122, and a coordinate projector 123. The camera 121 is configured to acquire near-infrared fluorescent signals and visible light signals. Among them, the cold light source illuminates the background during the acquisition process. For example, the reference parameters required for near-infrared light signal acquisition can be set as follows: at 800 nm, the quantum efficiency is higher than 30%, the frame rate is greater than 30 fps, and the image source (ie, the image source of the camera 121) is larger than 5 Micron. Preferably, a second band pass filter is placed between the camera 121 and the lens 122 to transmit near-infrared light having a wavelength of 810-870 nm. When the camera 121 is operated, the coordinate projector 123 can project a circular corridor to the detection area (not shown), which is the maximum range of the field of view, so that the operator can obtain the detection area of the system, and at the same time The operator obtains the excitation range of the multi-spectral light source module 110. As shown in FIG. 1, the head mounted system support module 130 can include a head mounted system bracket 131. The head mounted system bracket 131 is used to carry the light source module 110 and the signal acquisition module 120. Preferably, the head mounted system support module 130 may further include a voice recognition and control module 132. The voice recognition control module 132 can include A microphone, a voice recognition unit, and a control unit (not shown) are included to control the operation of the multi-spectral light source module 110, the coordinate projector 123, and the like by the voice of the operator. The speech recognition and control module 132 can be implemented using speech recognition techniques well known in the art. The visible light image and the near-infrared fluorescence image of the subject from the optical signal acquisition module 120 are input to the image processing module 140, respectively. The image processing module 140 is implemented by a back-end computer processing, and the acquisition and light source control can also be manually controlled by the back end. The image processing module 140 first pre-processes the input near-infrared fluorescence image to obtain a characteristic distribution of the fluorescence image based on the fluorescence specificity. Preprocessing may include noise removal, feature extraction, and dead point compensation. Of course, pre-processing well known in the art can also be performed on visible light images. Feature extraction can be performed on the input near-infrared fluorescence image using threshold segmentation. For example, for a pixel in the near-infrared fluorescence image where the image gray value G/background noise gray value is higher than 1.5, the gray value of the pixel is multiplied by 2 for pixels with G/G _n lower than 1.5. Point, the gray value of the pixel is divided by 2. According to this threshold segmentation method, the feature point can be strengthened. For the region of interest whose gray value is greater than the preset threshold, grayscale images to pseudo color known in the art can be used. The image adjustment algorithm converts the regions of interest into pseudo-color images, thereby further marking the locations of the feature points and the feature regions, so that the operator can perform the operations according to the images. The image processed by the image processing module 140 is a fused image, The general-purpose computer has a display and projection interface, which is convenient for the operator to realize the output display of the image. At the same time, the video signal can be fed back to the head-mounted system, and the fused image can be visualized by placing the front screen of the mirror.

Then, using the obtained optical characteristic distribution of the fluorescent image, the visible light image of the fluorescence input is image-fused, thereby obtaining a fusion result image for output. In particular, image fusion of a fluorescent image with a visible light image includes registration of the fluorescent image with the visible light image using the fluorescent image optical property distribution. This registration operation will be described in detail below.

The fluorescence image optical property distribution has fluorescence specificity, and the visible light image is a high resolution structural image. Image registration in accordance with embodiments of the present invention utilizes the above characteristics. In the registration, the morphological theory can be used to correct the minimized energy function of the optical image distribution of the fluorescence image so that its shape is close to the imaged structure. The following formula (1) can be used for registration.

In equation (1), J is a discrete Lapkce operator, U is a position vector, and n surface points are selected as the main marker points, respectively, the imaging surface marker points, = (p, .- a,.) motion vectors, Minimize ^) The vector f/ _{P is obtained} , then 3Ω _ρ = + is the position after surface deformation. In order to obtain a more accurate and high-resolution fused image, the image coincidence degree represented by the following formula (2) is used as a registration evaluation standard when performing registration.

Where Α is the normalized gray value matrix of the visible light image, and B is the normalized gray value matrix of the fluorescent image. The closer the result is to 1, the better the image registration effect. FIG. 3 shows a flow chart of an image processing method according to an embodiment of the present invention. As shown in FIG. 3, in step 301, spatial motion detection of the preprocessed visible light image sequence and the fluorescent image sequence is performed to filter out the mismatched minute displacement frames to obtain a visible light image sequence M1 and a fluorescent image sequence M2.

Optionally, in step 303, the image pyramid P1 is formed for the high-resolution visible light image sequence M1 obtained in step 301 to reduce the amount of data, thereby improving the real-time performance of the image processing. Specifically, the image is downsampled using a Gaussian pyramid to generate an i+1th layer from the i-th layer of the pyramid. First, the i-th layer is convolved with a Gaussian kernel, and then all even and even columns are deleted. Of course, the newly obtained image will become one-fourth the size of the previous image. In this case, the image is first expanded twice in each dimension, and the new row (even rows) is padded with 0. The convolution is then performed using the specified filter (actually a filter that is doubled in each dimension) to estimate the approximation of the "lost" pixel. The entire pyramid can be generated by performing an operation on the input image as described above. In step 305, edge detection is performed on the obtained image pyramid P1 and the fluorescence image sequence M2 by using a gradient edge detection method using, for example, a Roberts operator, to obtain image edges E1 and E2, respectively. Of course, in the case of high image processing capability, step 303 may be skipped to directly perform edge detection on the visible light image sequence M1 and the fluorescent image sequence M2.

At step 307, saliency-based sparse sampling is performed on the obtained image edges E1 and E2, respectively. The same method can be used to perform the saliency-based sparse sampling on the image edges E1 and E2 respectively. Here, the compressed sensing sparse sampling technique is used to sample the coefficients of E1 and E2, so that the sampling outputs S1 and S2 are obtained respectively.

At step 308, registration is performed on the sampled outputs S1 and S2 obtained in step 307. In addition to registration using equations (1) and (2) above, point cloud registration can be used to further optimize registration results. About For details of point cloud registration, please refer to "Xue Yaohong et al., Point Cloud Data Registration and Surface Subdivision Technology Research, National Defense Industry Press, 2011", which is not mentioned in this article.

Preferably, the image processing method according to the present invention may further include step 309. In step 309, algorithm convergence verification is performed on the result of the point cloud registration to ensure stable and reliable operation process.

Preferably, the processing of steps 301, 303, 305, and 309 can be performed by a smaller image GPU or FPGA while the central processing unit CPU with more computational power is used to perform the registration step 308 to further optimize system performance. At the same time, reduce the required hardware size.

The above described specific embodiments of the present invention are described in detail, and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Claims

Rights request

A head-mounted molecular image navigation system comprising:

a signal acquisition module, configured to acquire a near-infrared fluorescence image and a visible light image of the imaged object; a head mounted system support module, configured to carry the multi-spectral light source module and the signal acquisition module, to adjust the multi-spectral light source module pair The irradiation range of the detection area;

The image processing module is configured to perform image fusion on the collected near-infrared light image and the visible light image, and output the fused image.

2. The system according to claim 1, wherein the multi-spectral light source module comprises: a visible light source for emitting visible light to a subject;

a near-infrared laser for emitting near-infrared light to a subject; and

Light source coupler

Wherein the light source coupler couples the visible light and the near-infrared light, and couples the coupled light to the head mounted system support module through a single optical fiber.

3. The system according to claim 2, wherein the head mounted system support module comprises: a head mounted system bracket for carrying the multi-spectral light source module and the signal acquisition module; and a voice control module, Used to control the operation of the multispectral light source module to form a desired range of detection regions.

4. The system according to claim 1, wherein the image processing module performs feature extraction on the collected near-infrared fluorescence image, including:

For a pixel whose gray value G/background noise gray value is higher than 1.5, the gray value of the pixel is multiplied by 2; for a pixel whose G/Gn _is lower than 1.5, the pixel is The gray value of the point is divided by 2.

5. The system according to claim 4, wherein the image processing module performs image fusion on the acquired near-infrared fluorescent image and the visible light image, including obtaining optical characteristics of the near-infrared fluorescent image by minimizing an energy function formula: Distribution:

£(υ) = ||Δ /|| ² ₊ 3⁄4| /'− In equation (1), J is the discrete Lapkce operator, U is the position vector, select n surface points as the main marker points, respectively Surface marker point, = (p, - a,.) movement vector, obtained by minimizing The amount U _p , then 3 Ω _ρ = 3 Ω + is the position after surface deformation.

The system according to claim 1, wherein the image processing module performs image fusion on the collected near-infrared fluorescence image and the visible light image, including the image coincidence degree shown by the following formula as a registration effect evaluation criterion:

Where A is the normalized gray value matrix of the visible light image, and B is the normalized gray value matrix of the fluorescent image.

7. The system according to claim 5, wherein the image processing module further performs image fusion on the near-infrared fluorescent image and the visible light image by point cloud registration.

8. An image processing method applied to the head-mounted molecular image navigation system of claim 1, comprising:

Spatial motion detection of visible light image sequences and near-infrared fluorescent image sequences to filter out mismatched micro-displacement frames (301);

Downsampling the sequence of visible light images detected by spatial motion to obtain an image pyramid (303);

Edge detection is performed on the obtained image pyramid and near-infrared fluorescence image sequence by gradient edge detection method, and the image edge is obtained (305).

Performing saliency-based sparse sampling on the edges of the obtained images to obtain sampled outputs (307);

Registration is performed on the resulting sampled output for image fusion (308).

9. The method of claim 8, wherein the registering comprises obtaining an optical property distribution of the near-infrared fluorescent image by a minimum energy function:

£(υ) = ||Δ /|| ² ₊ 3⁄4| /'− In equation (1), J is the discrete Lapkce operator, U is the position vector, select n surface points as the main marker points, respectively Surface marker points, = (p, - a,.) move the vector, obtain the vector by minimizing; _P , then 3Ω _ρ = 3Ω + is the position after surface deformation.

10. The method according to claim 8, wherein the registration comprises using an image coincidence degree as shown in the following formula as a registration effect evaluation criterion:

Where A is a normalized gray value matrix of the visible light image, and B is a normalized gray value matrix of the near infrared fluorescent image.

11. The method of claim 9 further comprising using a point cloud registration to further image registration of the near infrared fluorescent image and the visible light image.

12. The method of claim 11 further comprising performing algorithm convergence verification on results of point cloud registration.