CN117649484A - Three-dimensional optical reconstruction method based on CT image - Google Patents

Abstract

The invention relates to a three-dimensional optical reconstruction method based on CT images, which includes: using a coaxial scanning device to obtain multi-angle CT projection signals and optical data of an object to be reconstructed; reconstructing CT projection signals from all angles to obtain a CT three-dimensional volume voxel data and extract the CT surface voxel data of the CT three-dimensional voxel data; align each pixel point in the optical data with the corresponding pixel point in the CT three-dimensional surface voxel data, and map the coordinates of the pixel points in the optical data to In the CT image coordinate system of the CT three-dimensional surface voxel data, the first optical data set with three-dimensional spatial coordinates in the CT image coordinate system at all angles is obtained; all optical data in this data set are spliced to form three-dimensional optically reconstructed information . The above method can generate accurate and comprehensive optical three-dimensional reconstructed objects by using CT scanning to provide accurate three-dimensional spatial information of the object for three-dimensional optical reconstruction.

Description

A three-dimensional optical reconstruction method based on CT images

Technical field

The invention belongs to the technical field of three-dimensional optical reconstruction, and in particular relates to a three-dimensional optical reconstruction method based on CT images.

Background technique

Three-dimensional reconstruction refers to converting two-dimensional images or projection data into spatial information of three-dimensional objects. The reconstructed model is convenient for computer display and further processing. It is used in medicine, biology, engineering, computer vision, etc. It has a wide range of applications in various fields and is of great significance for studying and analyzing the structure and form of objects. With the continuous advancement of imaging technology, high-resolution three-dimensional reconstruction has become an important research direction. Researchers improve algorithms, use high-resolution sensors and improve data acquisition methods to achieve more accurate and detailed 3D reconstruction results. In related technologies, three-dimensional reconstruction methods based on depth cameras can provide depth information of objects and therefore can be directly modeled. They mainly include structured light projection and TOF time-of-flight (Time offlight) methods. The structured light projection method uses a light source to project an image encoded according to certain rules to the object being measured. The encoding pattern is modulated by the shape of the object's surface and deforms. A camera is used to capture deformed structured light, and the depth information of the detected object is obtained through the positional relationship between the camera and the projection light source and the degree of deformation of the structured light. Although this method is highly accurate, it is easily interfered by ambient light around the object, and the structured light projector must be calibrated in advance. The TOF time-of-flight method calculates the depth distance of the surface of the measured object by recording the beam propagation time. The system transmitting device emits a pulse signal, which is received by the detector after being reflected by the object being measured. The depth value can be calculated based on the time from emission to reception of the optical signal and the speed of light. TOF technology may be limited by noise and accuracy at long distances or under low-light conditions. Its resolution is low and it cannot achieve high-precision three-dimensional reconstruction, and thus cannot accurately restore the shape and geometric structure of the object.

Contents of the invention

(1) Technical problems to be solved

In order to solve the above problems of the prior art, the present invention provides a three-dimensional optical reconstruction method based on CT images.

(2) Technical solutions

In order to achieve the above objectives, the main technical solutions adopted by the present invention include:

The present invention provides a three-dimensional optical reconstruction method based on CT images, including:

S10. Acquire multi-angle CT projection signals and optical data of the object to be reconstructed with the help of a coaxial scanning device; each of the multiple angles is the rotation angle of the rotating coaxial scanning device relative to the stationary object to be reconstructed; coaxial A CT imaging device and an optical imaging device having an installation angle with the CT imaging device are fixed in the scanning device;

S20. Reconstruct the CT projection signals from all angles according to the reconstruction algorithm to obtain the CT three-dimensional voxel data of the object to be reconstructed, and extract the surface voxel data of the CT three-dimensional voxel data of the object to obtain the CT three-dimensional surface voxel of the object to be reconstructed. data;

S30. Align and register each pixel point in the optical data with the corresponding pixel point in the CT three-dimensional surface voxel data, and map the coordinates of the pixel points in the aligned optical data to the CT of the CT three-dimensional surface voxel data. In the image coordinate system, obtain the first optical data set with three-dimensional spatial coordinates in the CT image coordinate system at all angles;

S40. Splice all the optical data in the first optical data set to form three-dimensional optically reconstructed information.

Optionally, the optical imaging device is a hyperspectral imaging device, then the optical data is hyperspectral data;

If the optical imaging device is a fluorescence imaging device, the optical data is fluorescence optical data;

If the optical imaging device is an RGB imaging device, the optical data is RGB optical data;

Multi-angle CT projection signals and optical data include: N CT projection signals and N optical data;

Among them, the rotating frame of the coaxial scanning device rotates relative to the object through an angle of Then the number of data generated when the coaxial scanning device rotates 360° relative to the object is/>

Optionally, in S20, CT projection signals from all angles are reconstructed according to the reconstruction algorithm to obtain CT three-dimensional voxel data of the object to be reconstructed, including:

The CT projection signals from each angle are preprocessed, and the filtered back-projection algorithm is used to reconstruct the preprocessed CT projection signals from all angles to obtain the CT three-dimensional voxel data of the object to be reconstructed.

Optionally, use the filtered back-projection algorithm to reconstruct the preprocessed CT projection signals from all angles to obtain the CT three-dimensional voxel data of the object to be reconstructed, including:

S21. Perform one-dimensional Fourier transform on the N preprocessed CT projection signals to obtain N first projection signals in the frequency domain;

S22. Filter the first projection signals in N frequency domains to obtain filtered N second projection signals;

S23. Perform one-dimensional inverse Fourier transform on the N second projection signals, restore them to the time domain, and obtain filtered N third projection signals in the time domain;

S24. Back-project each third projection signal. The back-projection is to evenly distribute the projection signal at each angle to each point passing through the object according to its original projection path, and project the same point on the object at all angles. The back-projection signals are accumulated to obtain the ray attenuation coefficient of each point of the object, and the CT three-dimensional voxel data of the object is reconstructed;

The CT three-dimensional voxel data includes: the three-dimensional spatial coordinates of each voxel in the CT image of the reconstructed object and the HU value of each voxel position. The HU value reflects the degree of X-ray absorption of the object to be reconstructed.

Optionally, extracting the surface voxel data of the CT three-dimensional voxel data of the object in S20 to obtain the CT three-dimensional surface voxel data of the object to be reconstructed also includes:

S25. Optimize the CT three-dimensional voxel data to obtain optimized CT three-dimensional voxel data;

S26. Extract the surface information of the CT three-dimensional voxel data from the optimized CT three-dimensional voxel data;

S27. Based on the preset voxel data threshold, divide the surface information of the CT three-dimensional voxel data into voxel data belonging to the object and voxel data belonging to the background;

S28. Based on the voxel data of the object, use traversal method to obtain the voxel data of the object boundary; set the HU value of the non-object boundary voxel data in the voxel data of the object to 0 to obtain the surface voxel data of the object, that is, CT 3D surface voxel data.

Optionally, the S30 includes:

The coaxial scanning device rotates relative to the object, and the angle between the imaging perspective and the object at each imaging position is The installation angle between the CT imaging equipment and the optical imaging equipment is θ;

The coordinate system of each imaging device of the coaxial scanning device is: the coordinate origin is located at the central axis of the rotating frame and is at the same height as the optical axis of the imaging device, the coordinate Z axis points from the coordinate origin to the center of each imaging device, and the X-ray imaging Z axis points from the coordinate origin X-ray center, the XY plane is perpendicular to the Z axis;

S31, for every imaging angle For the surface voxel data under In the plane, each pixel point in the plane is the projection position of the three-dimensional surface voxel data on the projection plane, forming the CT two-dimensional projection image (x, y, HU) at this angle; to obtain all imaging angles/> CT two-dimensional projection image below;

S32. Regarding imaging angle CT two-dimensional projection images and imaging angles/> Perform feature detection on the optical data under the CT two-dimensional projection image and obtain the significant feature points in the optical data;

S33. Obtain the feature descriptors of respective significant feature points and perform matching, and obtain matching feature points that exceed the preset threshold;

S34. Obtain the imaging perspective based on the matched feature point pairs. CT two-dimensional projection image and imaging perspective Spatial coordinate transformation mapping of optical data;

S35. Convert the imaging perspective based on spatial coordinate conversion mapping CT two-dimensional projection image and imaging perspective Perform registration with the optical data below;

Based on the method from S32 to S35, all imaging angles are traversed Align and register CT two-dimensional projection images and optical data from all imaging angles.

Optionally, the S34 includes:

S341. Based on each matching feature point pair, divide the coordinates of the feature points belonging to each imaging device by the focal length of the imaging device to obtain the normalized feature point pair coordinates;

S342. Construct a linear equation based on the normalized feature point pair coordinates;

Set p(x,y) and p’(x’,y’) to be the normalized feature point pair coordinates;

p(x,y) corresponds to the CT two-dimensional projection image, p’(x’,y’) corresponds to the optical data;

Determine the basic matrix through the linear equation p′ ^T Fp=0, where F is the basic matrix;

Set the linear equation constructed by all pairs of feature points to solve the basic matrix;

S343. Based on the basic matrix, use the internal parameters of the CT imaging equipment and the optical imaging equipment to perform triangulation, and map the normalized feature point pair coordinates to three-dimensional points in the world coordinate system;

S344. Use the three-dimensional point coordinates mapped by the normalized feature point pairs in the world coordinate system to obtain the spatial coordinate transformation mapping. The spatial coordinate transformation mapping includes a translation vector and a rotation matrix.

Optionally, the S34 includes:

Convert the pixel positions in the optical data at each angle into coordinates in the registered CT two-dimensional projection image coordinate system;

Establish a correspondence between the pixels in the optical data and the spatial positions in the CT three-dimensional voxel data, map the pixel information in the optical data at each angle to the spatial positions of the CT three-dimensional surface voxel data, and obtain all angles The first optical data set with three-dimensional spatial coordinates.

Optionally, the S40 includes:

The S30 method is used to traverse all adjacent optical data, register the adjacent optical data, identify overlapping areas, and splice all optical data in the first optical data set based on the identified overlapping areas to form three-dimensional optically reconstructed information. ;

Optimize the three-dimensional optically reconstructed information to obtain complete three-dimensional optically reconstructed information.

In a second aspect, the present invention also provides a computing device, including: a memory and a processor, the memory stores a computer program, the processor executes the computer program in the memory, and executes any of the above-mentioned first aspects. The steps of the three-dimensional optical reconstruction method based on CT images are described.

(3) Beneficial effects

The three-dimensional optical reconstruction method of the present invention does not require pre-calibration in advance, is less affected by ambient light, and improves the accuracy of three-dimensional optical data reconstruction. CT images have high spatial resolution and can provide the fine structure and accurate three-dimensional spatial coordinates of the object. This enables the three-dimensional optical reconstruction method based on CT images to achieve high-precision three-dimensional optical reconstruction and accurately restore the objects in the optical image. Shape and geometric structure, especially suitable for three-dimensional optical reconstruction of objects with drastic surface changes, that is, drastic depth changes.

Description of drawings

Figure 1 is a schematic flow chart of a three-dimensional optical reconstruction method based on CT images provided by the present invention;

Figure 2(a) and Figure 2(b) are both schematic diagrams of the coaxial scanning device provided by the present invention;

Figure 3(a) shows the rotation of the imaging device relative to the object Schematic diagram of X-ray imaging at angle;

Figure 3(b) is a schematic diagram of the reconstructed CT three-dimensional voxel data containing the object and background, segmenting the object background and extracting the surface of the CT surface voxel data;

Figure 3(c) is a schematic diagram of a CT two-dimensional projection image obtained by two-dimensional projection of CT surface voxel data;

Figure 3(d) shows the rotation of the imaging device relative to the object Schematic diagram of optical imaging at angle;

Figure 3(e) is a schematic diagram of the collected optical image data;

Figure 3(f) shows the CT two-dimensional projection image under angle and/> Schematic diagram of registration of optical data at different angles.

Detailed ways

In order to better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention may be understood more clearly and thoroughly, and the scope of the present invention may be fully conveyed to those skilled in the art.

The three-dimensional optical reconstruction method in the embodiment of the present invention does not belong to the optical reconstruction of human body images in the medical field. The objects to be reconstructed in the embodiment of the present invention may be larger plants or other food plants.

Embodiment 1

As shown in Figures 1 to 3, the present invention provides a three-dimensional optical reconstruction method based on CT images. The method is executed by any computing device and includes the following steps:

S10. Acquire multi-angle CT projection signals and optical data of the object to be reconstructed with the help of a coaxial scanning device; each of the multiple angles is the rotation angle of the rotating coaxial scanning device relative to the stationary object to be reconstructed; coaxial A CT imaging device and an optical imaging device having an installation angle with the CT imaging device are fixed in the scanning device. The computing device of this embodiment can be electrically connected to the CT imaging device and the optical imaging device. In other embodiments, the processing functionality of the computing device may be integrated with the CT imaging device or integrated with the optical imaging device.

The optical data in this step can be understood as optical images. In this embodiment, optical data are used for explanation.

For example, if the optical imaging device is a hyperspectral imaging device, then the optical data is hyperspectral data;

If the optical imaging device is a fluorescence imaging device, the optical data is fluorescence optical data. The above-mentioned optical imaging device may be an RGB imaging device, and the optical data may also be RGB optical data.

In this embodiment, the multi-angle CT projection signals and optical data include: N CT projection signals and N optical data;

Among them, the rotating frame of the coaxial scanning device rotates relative to the object through an angle of Then the number of data generated when the coaxial scanning device rotates 360° relative to the object is/> N is a natural number greater than or equal to 1. Preferably, N is a natural number equal to or greater than 10. In this embodiment, N can be 360, that is, the rotation angle of the rotating frame relative to the object is from 0° to 360°, and the data at each angle are recorded as/> For example, each time the rotation angle is 1°, the number of data is 360. Different angle accuracy results in different CT imaging resolutions.

S20. Reconstruct the CT projection signals from all angles according to the reconstruction algorithm to obtain the CT three-dimensional voxel data of the object to be reconstructed, and extract the surface voxel data of the CT three-dimensional voxel data of the object to obtain the CT three-dimensional surface voxel of the object to be reconstructed. data.

Usually, the CT projection signals from each angle are preprocessed, and a filtered back-projection algorithm is used to reconstruct the preprocessed CT projection signals from all angles to obtain the CT three-dimensional voxel data of the object to be reconstructed, and then extract the CT three-dimensional voxel data. The surface information of the voxel data is obtained to obtain the CT three-dimensional surface voxel data of the object to be reconstructed.

S30. Align and register each pixel point in the optical data with the corresponding pixel point in the CT three-dimensional surface voxel data, and map the coordinates of the pixel points in the aligned optical data to the CT three-dimensional surface voxel data. In the CT image coordinate system, a first optical data set with three-dimensional spatial coordinates in the CT image coordinate system at all angles is obtained.

S40. Splice all the optical data in the first optical data set to form three-dimensional optically reconstructed information.

The method of this embodiment does not require pre-calibration in advance, is less affected by ambient light, and improves the accuracy of three-dimensional optical data reconstruction. CT images have high spatial resolution and can provide the fine structure and accurate three-dimensional spatial coordinates of the object. This enables the three-dimensional optical reconstruction method based on CT images to achieve high-precision three-dimensional optical reconstruction and accurately restore the objects in the optical image. Shape and geometric structure, especially suitable for three-dimensional optical reconstruction of objects with drastic surface changes, that is, drastic depth changes.

In order to better understand the above-mentioned processes of step S20 and step S30, step S20 and step S30 will be explained step by step below.

The process for step S20 includes the following sub-steps:

S21. Perform one-dimensional Fourier transform on the N preprocessed CT projection signals to obtain N first projection signals in the frequency domain;

S22. Filter the first projection signals in N frequency domains to obtain filtered N second projection signals;

S23. Perform one-dimensional inverse Fourier transform on the N second projection signals, restore them to the time domain, and obtain filtered N third projection signals in the time domain;

S24. Back-project each third projection signal. Back-projection is to evenly distribute the projection signal at each angle to each point passing through the object according to its original projection path, and project all angles on the object at the same time. The back-projection signals of one point are accumulated to obtain the ray attenuation coefficient of each point of the object, and the CT three-dimensional voxel data of the object is reconstructed.

In other words, a CT three-dimensional voxel data accumulated from all angles.

The CT three-dimensional voxel data includes: the three-dimensional spatial coordinates of each voxel in the CT image of the reconstructed object and the HU value (Hounsfiled Unit value) of each voxel position. The HU value reflects the degree of X-ray absorption of the object to be reconstructed.

In this field, data grid points in three-dimensional images are called voxels, and in two-dimensional images they are called pixels. The following sub-steps are detailed descriptions of extracting the surface voxel data of the CT three-dimensional voxel data of the object in S20 and obtaining the CT three-dimensional surface voxel data of the object to be reconstructed.

S25. Optimize the CT three-dimensional voxel data to obtain optimized CT three-dimensional voxel data;

S26: Extract surface information of the CT three-dimensional voxel data from the optimized CT three-dimensional voxel data.

For example, for the above CT voxel data, select an appropriate threshold that can distinguish object voxels from background voxels. This threshold will be used to divide the voxels into two parts: object voxel data and background voxel data; use select The threshold of divides the voxel data into two areas: one is the voxel data of the object, and the other is the voxel data of the background, as recorded in sub-step S27.

S27. Based on the preset voxel data threshold, divide the CT three-dimensional voxel data into voxel data belonging to the object and voxel data belonging to the background;

S28. Based on the voxel data of the object, use traversal method to obtain the voxel data of the object boundary;

That is, traverse the voxel data of all objects, and for each voxel data, check whether its adjacent voxel data belongs to the background voxel data. If there is background voxel data in the adjacent voxel data of a certain voxel data, then This voxel data is right on the boundary of the object;

S29. Set the HU value of the non-object boundary voxel data in the object voxel data to 0 to obtain the object's surface voxel data, that is, the CT three-dimensional surface voxel data.

For example, the HU value of the voxel data that is not at the boundary is set to 0, and the HU value of the boundary voxel data remains unchanged. The three-dimensional voxel data formed in this way only contains the object surface voxel data.

Through the above sub-steps S21 to sub-step S29, a process of obtaining CT three-dimensional voxel data of the object to be reconstructed and obtaining CT three-dimensional surface voxel data in step S20 is described. In other embodiments, other methods can be used to implement it. , this embodiment does not limit it.

In this embodiment, the coaxial scanning device rotates relative to the object, and the angle between the imaging angle of each imaging position and the object is The installation angle between the CT imaging equipment and the optical imaging equipment is θ;

(the angle turned Refers to each imaging position. If there are N imaging positions, then there are N/> N imaging data);

In this embodiment, the coordinate system of each imaging device is defined as follows: the coordinate origin is located at the central axis of the rotating frame and is at the same height as the optical axis of the imaging device, the coordinate Z axis points from the coordinate origin to the center of the imaging device, and the X-ray imaging Z axis points from the coordinate origin. X-ray center, the XY plane is perpendicular to the Z axis.

Correspondingly, the process of step S30 includes the following sub-steps:

S31, for every imaging angle For the surface voxel data under In the plane, each pixel point in the plane is the projection position of the three-dimensional surface voxel data on the projection plane, forming a two-dimensional CT projection image (x, y, HU) at this angle; to obtain all imaging angles/> The two-dimensional CT projection image below.

S32. Regarding imaging angle CT two-dimensional projection images and imaging angles/> Perform feature detection on the optical data under the CT two-dimensional projection image and obtain the significant feature points in the optical data;

S33. Obtain the feature descriptors of respective significant feature points and perform matching, and obtain matching feature point pairs that exceed the preset threshold;

S34. Obtain the imaging perspective based on the matched feature point pairs. CT two-dimensional projection image and imaging perspective Spatial coordinate conversion mapping of optical data;

This sub-step S34 also includes the following sub-steps:

S341. Based on each matching feature point pair, divide the coordinates of the feature points belonging to each imaging device by the focal length of the imaging device to obtain the normalized feature point pair coordinates;

S342. Construct a linear equation based on the normalized feature point pair coordinates;

Set p(x,y) and p’(x’,y’) to be the normalized feature point pair coordinates;

p(x,y) corresponds to the CT two-dimensional projection image, p’(x’,y’) corresponds to the optical data;

Through the linear equation p′ ^T Fp=0, determine the basic matrix, where F is the basic matrix and T is the transpose;

Set the linear equation constructed by all pairs of feature points to solve the basic matrix;

S343. Based on the basic matrix, use the internal parameters of the imaging device to which the CT two-dimensional projection image belongs and the imaging device to which the optical data belongs to perform triangulation, and map the normalized feature point pair coordinates to a three-dimensional point in the world coordinate system;

S344. Use the three-dimensional point coordinates mapped by the normalized feature point pairs in the world coordinate system to obtain the spatial coordinate transformation mapping. The spatial coordinate transformation mapping includes a translation vector and a rotation matrix.

S35. Convert the imaging perspective based on spatial coordinate conversion mapping CT two-dimensional projection image and imaging perspective Perform registration with the optical data below;

Based on the method of sub-step S32 to sub-step S35, all imaging angles are traversed Align and register CT two-dimensional projection images and optical data from all imaging angles.

Convert the pixel positions in the optical data at each angle into coordinates in the registered CT two-dimensional projection image coordinate system;

Establish a correspondence between the pixels in the optical data and the spatial positions in the CT three-dimensional voxel data, map the pixel information in the optical data at each angle to the spatial positions of the CT three-dimensional surface voxels, and obtain the spatial positions of the CT three-dimensional surface voxels at all angles. The first optical data set with three-dimensional spatial coordinates.

The above process is not only applicable to hyperspectral images, but also uses fluorescence optical data, RGB optical data, etc. This embodiment is not limited thereto. The optical imaging data of the coaxial scanning device is configured according to actual needs, and then the corresponding optical data is obtained.

Embodiment 2

The three-dimensional optical reconstruction method based on CT images in this embodiment is shown in detail in conjunction with Figure 2(a), Figure 2(b), Figure 3(a) to Figure 3(f). The optical data in this embodiment is a fluorescence image. or other optical images. The method of this embodiment may include the following steps:

201. First, use a coaxial scanning device to obtain CT projection signals and fluorescence data from multiple angles of the object to be reconstructed. The object is relatively motionless during the process of obtaining three-dimensional data. There is a CT projection signal and fluorescence data at each angle; the multiple angles can be the angles at which the imaging equipment in the coaxial scanning device rotates around the object to be reconstructed.

The coaxial scanning device of this embodiment includes: a rotating frame, CT imaging equipment (ie, X-ray source, X-ray detector), and fluorescence imaging equipment (ie, light source and camera) fixed on the rotating frame.

Figure 2(a) and Figure 2(b) show the schematic diagram of the coaxial scanning device, where XYZ is the object coordinate system and X’Y’Z is the rotating frame coordinate system. CT imaging equipment (including X-ray source and ray detector, the Other optical imaging equipment, and is not limited to one type, can also have multiple optical imaging equipment at the same time. Here, a fluorescence imaging camera is taken as an example) installed on a ring rotating frame, the center height of the imaging equipment is consistent, the X-ray source and the fluorescence imaging camera The installation angle is θ, and the object to be reconstructed is located in the center of the annular rotating frame.

During imaging, the object does not move, and the rotation of the rotating frame drives the imaging equipment on it to rotate around the object (that is, the object coordinate system remains unchanged, and the rotating frame coordinate system rotates around the Z axis). Each time it rotates through a certain angle (such as The smaller the angle, the higher the accuracy of CT reconstruction.) Complete one imaging. Imaging includes:

1) CT imaging at this angle mainly refers to the ray intensity signal received by the ray detector after the X-rays are attenuated by the object at this angle, which is called the CT projection signal at this angle;

2) Fluorescence imaging at this angle is the optical image data used for reconstruction. One rotation completes the acquisition of CT projection signals and optical image data at all angles. Assuming that imaging is performed every 1° of rotation, when the rotating frame rotates and scans 360° with the device, 360 CT projection signals and 360 fluorescence data can be obtained. .

202. Data preprocessing.

CT projection signals are preprocessed to reduce noise and enhance image quality.

This data preprocessing can be implemented using existing methods, such as preprocessing such as artifact removal, gamma correction, and filtering. This embodiment does not limit it, and it can be selected according to actual needs.

203. Use the reconstruction algorithm to reconstruct the CT projection signals from all angles to obtain the CT three-dimensional voxel data of the object to be reconstructed.

The essence of CT reconstruction is to use the CT projection signal obtained in step 201 to solve the X-ray attenuation coefficient distribution inside the object (that is, the X-ray attenuation coefficient of different parts of the object. Different materials have different attenuation of X-rays, and CT imaging is based on this to achieve non-destructive Detecting the distribution of materials inside an object).

In this embodiment, a filtered back projection algorithm (Filtered Back Projection, FBP) is used. The reconstruction steps of the FBP algorithm are:

1) Perform one-dimensional Fourier transform on the 360 CT projection signals in the time domain to obtain 360 projection signals in the frequency domain;

2) Filter the 360 CT projection signals in the frequency domain to obtain 360 filtered CT projection signals;

3) Perform one-dimensional inverse Fourier transform on the 360 filtered CT projection signals and restore them to the time domain to obtain the 360 filtered CT projection signals in the time domain;

4) Back-project each filtered projection signal; accumulate the back-projection signals at 360 angles to calculate the attenuation of each part of the object, that is, reconstruct a three-dimensional voxel data of the object.

Back projection is to evenly distribute the projection signal at each angle to each point passing through the object according to its original projection path. The three-dimensional voxel data of the object includes the three-dimensional spatial coordinates and the attenuation of X-rays by the object within the voxel. coefficient.

All three-dimensional voxels form the three-dimensional spatial shape of the object, which will be used for subsequent three-dimensional optical reconstruction of the object to provide the accurate three-dimensional spatial coordinates of the object.

204. Perform subsequent processing on the three-dimensional voxel data of the object obtained through CT reconstruction, including denoising and contrast enhancement, to obtain CT three-dimensional voxel data with improved quality.

Denoising is to reduce noise and artifacts in images to improve the quality of reconstructed data; enhancing contrast can improve the readability of reconstructed data.

205. Extract the surface information of the CT three-dimensional voxel data from the CT three-dimensional voxel data of the object obtained in step 204.

Based on the preset voxel data threshold, the CT three-dimensional voxel data is divided into voxel data belonging to the object and voxel data belonging to the background; based on the voxel data belonging to the object, a traversal method is used to obtain the voxel data of the object boundary; Set the HU value of the non-object boundary voxel data in the voxel data to which the object belongs to 0, and retain the HU value of the voxel data at the object interface as the original HU value, to obtain the surface voxel data of the object, that is, the CT three-dimensional surface voxel data.

206. Register the CT three-dimensional surface voxel data of the object obtained in step 205 and the fluorescence data obtained in step 201.

Generally, registration refers to mapping two or more images from two or more sets of image data acquired at different times or by different imaging devices by finding a spatial transformation to map one image to another image. This makes the points corresponding to the same position in space in the two images correspond one to one.

Since the CT imaging equipment and the fluorescence imaging equipment are both installed on the rotating frame, the angle between them in the same three-dimensional coordinate system (the coordinate system of the rotating frame) is θ, and the rotating frame rotates relative to the object. CT imaging angle (as shown in Figure 2(a)) and rotation angle/> The imaging angle (as shown in Figure 2(b)) corresponds to the shooting of the same part of the object.

The steps for registering the CT three-dimensional surface voxel data and fluorescence data reconstructed in step 205 are as follows:

1) The coordinate system of each imaging device of the coaxial scanning device is: the coordinate origin is located at the central axis of the rotating frame and is at the same height as the optical axis of the imaging device, the coordinate Z axis points from the coordinate origin to the center of each imaging device, and the XY plane is perpendicular to the Z axis; For every imaging angle For the surface voxel data under In the plane, each pixel point in the plane is the projection position of the three-dimensional surface voxel data on the projection plane, forming a two-dimensional CT projection image (x, y, HU) at this angle; to obtain all imaging angles/> The two-dimensional CT projection image below.

2) For the angle The following two-dimensional CT projection images and corresponding angles/> Feature detection is performed manually or automatically on the optical image below to obtain significant feature points in the CT two-dimensional projection image and optical image;

For example, feature detection can be performed manually or automatically to find significant feature points in the image (which can be edges, intersections, contours, shapes, structures, etc.). Manual detection methods involve marking features of interest on the image. , suitable for situations where specific features need to be selected with high accuracy. Automatic feature detection uses computer vision libraries such as OpenCV, and can effectively and quickly locate feature points by calling appropriate feature detection algorithms such as Harris corner detection, SIFT, SURF, FAST, ORB, etc.;

3) Calculate the angle The following two-dimensional CT projection images and corresponding angles/> The feature descriptors of the feature points found in the optical images under Feature points (for example, using nearest neighbor matching to match each feature point in one image with the closest feature point in another image). After completing the matching, the difference between the descriptors of the feature points of the two images is quantified. Similarity is used to determine the quality of matching. The similarity measure can be to calculate the Euclidean distance between descriptor vectors. The smaller the distance, the more similar it is, which further indicates that the feature point matching is better.

Thereby, the corresponding salient features between the two-dimensional CT projection image and the corresponding fluorescence image are found. By comparing the distance measures between feature descriptors, the most matching descriptor pair is selected to achieve feature point matching between images;

4) Calculate the spatial coordinate transformation relationship between the two images through matching feature point pairs; the specific steps are as follows: (1) For each feature point pair, divide the coordinates of the feature point by the focal length of the imaging device for normalization ; (2) Construct a linear equation for the normalized coordinates of each feature point pair. Assume that p(x,y) and p'(x',y') are the normalized coordinates of any pair of matching points in the CT two-dimensional projection image and the optical image. The basic matrix is given by the linear equation p′ ^T Fp= 0 definition, where F is the basic matrix, combining all angles The following two-dimensional CT projection images and corresponding angles/> The basic matrix is solved by solving the linear equation constructed by all pairs of feature points in the optical image, and the basic matrix describing the geometric relationship between the two imaging devices (CT imaging detector and optical imaging camera) is obtained. (3) Use the internal parameters of the CT imaging detector and the optical imaging camera to perform triangulation and map the feature point pairs to three-dimensional points in the world coordinate system. (4) Using the three-dimensional point coordinates of the feature point pair in the world coordinate system, calculate the spatial coordinate transformation relationship between the CT two-dimensional projection image and the optical image, including the translation vector and rotation matrix.

That is, the angle is calculated through the matching feature point pairs The following two-dimensional CT projection images and corresponding angles The spatial coordinate transformation relationship between optical images (such as translation, rotation, scaling, etc.), in the same way, traverse all Complete the above operations 2) to 4) for all imaging angles, and finally register the two images through the calculated spatial coordinate transformation relationship. Realize the alignment between the pixels of the CT two-dimensional projection image and the optical image at each angle. That is, the alignment between the pixel points of the CT two-dimensional projection image and the optical data is achieved at all imaging angles.

207. Coordinate mapping.

First, the coordinates of the optical image pixels are mapped into the CT image coordinate system.

Convert the pixel positions in the optical data at each angle into coordinates in the registered CT two-dimensional projection image coordinate system; establish the corresponding relationship between the pixels in the optical data and the spatial positions in the CT three-dimensional voxel data, The image pixel information in the optical data at each angle is mapped to the spatial position of the CT three-dimensional surface voxel, and an optical data set with three-dimensional spatial coordinates at all angles is obtained.

208. Since the optical images at adjacent angles have sufficient overlap, the optical data with three-dimensional spatial coordinates at two adjacent angles can be registered through the registration method used in step 205, so as to splice and reconstruct a complete image. 3D optical image.

209. Three-dimensional reconstruction post-processing. To further optimize the generated three-dimensional optical image, Gaussian filtering can be used to make the splicing smoother and smoother to achieve a more realistic three-dimensional optical reconstruction.

In practical applications, the filtering parameters are fine-tuned according to the image characteristics and needs to obtain the best results.

The method of this embodiment can generate an accurate and comprehensive three-dimensional model by utilizing the high-resolution and multi-layer information provided by CT scans. This provides powerful tools and resources for visualization, analysis and application.

This embodiment has the advantage of high resolution, that is, the CT image has a high spatial resolution, can capture the minute details and structure of the object, and can accurately express the shape and characteristics of the object. By inputting data through CT image crops, the real coordinate information of the object's three-dimensional space can be provided, and the three-dimensional space positioning prepared for the three-dimensional optical reconstruction of the object can be provided, which helps to generate a highly accurate three-dimensional optical model.

In addition, an embodiment of the present invention also provides a computing device, including: a memory and a processor, the memory stores a computer program, the processor executes the computer program in the memory, and executes the steps described in any of the above embodiments. The steps of a three-dimensional optical reconstruction method based on CT images.

In the description of this specification, the terms "one embodiment", "some embodiments", "embodiments", "examples", "specific examples" or "some examples", etc., refer to the description in conjunction with the embodiment or example. A specific feature, structure, material, or characteristic described is included in at least one embodiment or example of the invention. In this specification, the schematic expressions of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in a combined manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine different embodiments or examples and features of different embodiments or examples described in this specification unless they are inconsistent with each other.

Although the embodiments of the present invention have been shown and described above, it can be understood that the above-mentioned embodiments are illustrative and should not be construed as limitations of the present invention. Those of ordinary skill in the art can make modifications to the above-mentioned embodiments within the scope of the present invention. The embodiments are subject to alterations, modifications, substitutions and variations.

Claims (10)

1. A three-dimensional optical reconstruction method based on CT images, characterized by including: S10. Acquire multi-angle CT projection signals and optical data of the object to be reconstructed with the help of a coaxial scanning device; each of the multiple angles is the rotation angle of the rotating coaxial scanning device relative to the stationary object to be reconstructed; coaxial A CT imaging device and an optical imaging device having an installation angle with the CT imaging device are fixed in the scanning device; S20. Reconstruct the CT projection signals from all angles according to the reconstruction algorithm to obtain the CT three-dimensional voxel data of the object to be reconstructed, and extract the surface voxel data of the CT three-dimensional voxel data of the object to obtain the CT three-dimensional surface voxel of the object to be reconstructed. data; S30. Align and register each pixel point in the optical data with the corresponding pixel point in the CT three-dimensional surface voxel data, and map the coordinates of the pixel points in the aligned optical data to the CT image of the CT three-dimensional voxel data. In the coordinate system, obtain the first optical data set with three-dimensional spatial coordinates in the CT image coordinate system at all angles; S40. Splice all the optical data in the first optical data set to form three-dimensional optically reconstructed information. 2. The method of claim 1, wherein the optical imaging device is a hyperspectral imaging device, and the optical data is hyperspectral data; If the optical imaging device is a fluorescence imaging device, the optical data is fluorescence optical data; if the optical imaging device is an RGB imaging device, the optical data is RGB optical data; Multi-angle CT projection signals and optical data include: N CT projection signals and N optical data; Among them, the rotating frame of the coaxial scanning device rotates relative to the object through an angle of Then the number of data generated when the coaxial scanning device rotates 360° relative to the object is/> 3. The method according to claim 1, characterized in that, in S20, CT projection signals from all angles are reconstructed according to the reconstruction algorithm to obtain CT three-dimensional voxel data of the object to be reconstructed, including: The CT projection signals from each angle are preprocessed, and the filtered back-projection algorithm is used to reconstruct the preprocessed CT projection signals from all angles to obtain the CT three-dimensional voxel data of the object to be reconstructed. 4. The method according to claim 3, characterized in that a filtered back-projection algorithm is used to reconstruct the pre-processed CT projection signals from all angles to obtain the CT three-dimensional voxel data of the object to be reconstructed, including: S21. Perform one-dimensional Fourier transform on the N preprocessed CT projection signals to obtain N first projection signals in the frequency domain; S22. Filter the first projection signals in N frequency domains to obtain filtered N second projection signals; S23. Perform one-dimensional inverse Fourier transform on the N second projection signals, restore them to the time domain, and obtain filtered N third projection signals in the time domain; S24. Back-project each third projection signal. The back-projection is to evenly distribute the projection signal at each angle to each point passing through the object according to its original projection path, and project the same point on the object at all angles. The back-projection signals are accumulated to obtain the ray attenuation coefficient of each point of the object, and the CT three-dimensional voxel data of the object is reconstructed; The CT three-dimensional voxel data includes: the three-dimensional spatial coordinates of each voxel in the CT image of the reconstructed object and the HU value of each voxel position. The HU value reflects the degree of X-ray absorption of the object to be reconstructed. 5. The method according to claim 4, characterized in that, in the step S20, the surface voxel data of the CT three-dimensional voxel data of the object is extracted to obtain the CT three-dimensional surface voxel data of the object to be reconstructed, including: S25. Optimize the CT three-dimensional voxel data to obtain optimized CT three-dimensional voxel data; S26. Extract the surface information of the CT three-dimensional voxel data from the optimized CT three-dimensional voxel data; S27. Based on the preset voxel data threshold, divide the surface information of the CT three-dimensional voxel data into voxel data belonging to the object and voxel data belonging to the background; S28. Based on the voxel data of the object, use traversal method to obtain the voxel data of the object boundary; set the HU value of the non-object boundary voxel data in the voxel data of the object to 0 to obtain the surface voxel data of the object, that is, CT 3D surface voxel data. 6. The method according to claim 5, characterized in that said S30 includes: The coaxial scanning device rotates relative to the object, and the angle between the imaging perspective and the object at each imaging position is The installation angle between the CT imaging equipment and the optical imaging equipment is θ; The coordinate system of each imaging device of the coaxial scanning device is: the coordinate origin is located at the central axis of the rotating frame and is at the same height as the optical axis of the imaging device, the coordinate Z axis points from the coordinate origin to the center of each imaging device, and the X-ray imaging Z axis points from the coordinate origin X-ray center, XY plane is perpendicular to Z axis; S31, for every imaging angle For surface voxel data under In the plane, each pixel point in the plane is the projection position of the three-dimensional surface voxel data on the projection plane, forming the CT two-dimensional projection image (x, y, HU) at this angle; to obtain all imaging angles/> CT two-dimensional projection image below; S32. Regarding imaging angle CT two-dimensional projection images and imaging angles/> Perform feature detection on the optical data under the CT two-dimensional projection image and obtain the significant feature points in the optical data; S33. Obtain the feature descriptors of respective significant feature points and perform matching, and obtain matching feature point pairs that exceed the preset threshold; S34. Obtain the imaging perspective based on the matched feature point pairs. CT two-dimensional projection images and imaging angles/> Spatial coordinate transformation mapping of optical data; S35. Convert the imaging perspective based on spatial coordinate conversion mapping CT two-dimensional projection images and imaging angles/> Perform registration with the optical data below; Based on the method from S32 to S35, all imaging angles are traversed Align and register CT two-dimensional projection images and optical data from all imaging angles. 7. The method according to claim 6, characterized in that said S34 includes: S341. Based on each matching feature point pair, divide the coordinates of the feature points belonging to each imaging device by the focal length of the imaging device to obtain the normalized feature point pair coordinates; S342. Construct a linear equation based on the normalized feature point pair coordinates; Set p(x,y) and p’(x’,y’) to be the normalized feature point pair coordinates; p(x,y) corresponds to the CT two-dimensional projection image, p’(x’,y’) corresponds to the optical data; Determine the basic matrix through the linear equation p′ ^T Fp=0, where F is the basic matrix; Set the linear equation constructed by all pairs of feature points to solve the basic matrix; S343. Based on the basic matrix, use the internal parameters of the CT imaging equipment and the optical imaging equipment to perform triangulation, and map the normalized feature point pair coordinates to three-dimensional points in the world coordinate system; S344. Use the three-dimensional point coordinates mapped by the normalized feature point pairs in the world coordinate system to obtain the spatial coordinate transformation mapping. The spatial coordinate transformation mapping includes a translation vector and a rotation matrix. 8. The method according to claim 6, characterized in that said S34 includes: Convert the pixel positions in the optical data at each angle into coordinates in the registered CT two-dimensional projection image coordinate system; Establish a correspondence between the pixels in the optical data and the spatial positions in the CT three-dimensional surface voxel data, map the pixel information in the optical data at each angle to the spatial positions of the CT three-dimensional surface voxel data, and obtain all The first optical data set with three-dimensional spatial coordinates under angle. 9. The method according to claim 8, characterized in that said S40 includes: The S30 method is used to traverse all adjacent optical data, register the adjacent optical data, identify overlapping areas, and splice all optical data in the first optical data set based on the identified overlapping areas to form three-dimensional optically reconstructed information. ; Optimize the three-dimensional optically reconstructed information to obtain complete three-dimensional optically reconstructed information. 10. A computing device, characterized in that it includes: a memory and a processor, a computer program is stored in the memory, the processor executes the computer program in the memory, and executes any one of the above claims 1 to 9. The steps of the three-dimensional optical reconstruction method based on CT images are described.