CN104615739A - A fast data archiving method suitable for high-resolution massive atlases of 3D brain tissue - Google Patents

Abstract

The invention relates to a data rapid filing method suitable for a three-dimensional brain tissue high-resolution mass atlas, which comprises the following steps: generating the archived data: dividing a three-dimensional data space into a plurality of data blocks with equal size, generating n +1 sets of data sets with n +1 different resolutions, and storing the data sets as independent data documents according to a path self-index structure; extracting the archived data: calculating the minimum data block required by the interested area through the start coordinate and the end coordinate of the interested area and the 6 parameters of the several-level resolution; and sequentially reading the independent data documents of the data blocks required by the region of interest into the memory, and forming an interested data space with the data points of the reserved region of interest. The invention archives the three-dimensional brain tissue high-resolution massive atlas, and can quickly and continuously take out the interested data blocks from the archived file without an index file. The document establishing and extracting process has parallelism, and can be further accelerated by applying parallel calculation.

Description

A fast data archiving method suitable for high-resolution massive atlases of 3D brain tissue

technical field

The present invention is suitable for the field of biomedical image processing, more specifically, it is suitable for archiving high-resolution massive atlases of three-dimensional brain tissue, and can quickly and continuously retrieve interested data blocks from archived files without increasing hardware investment. Under the circumstances, the processing ability of the computer system for the three-dimensional massive image data sets obtained in the brain science research is greatly improved, and it specifically relates to a fast data archiving method suitable for high-resolution massive atlases of three-dimensional brain tissue.

Background technique

In current brain science research, with the development of imaging technology, on the one hand, researchers can image a wide range of brain tissue, and on the other hand, they can image finer brain tissue. With the improvement of imaging range and imaging precision The amount of image data generated is also very large, often reaching TB level. How to deal with these massive brain tissue image data is a very big challenge. There are usually two solutions. The first is to improve the level of hardware, such as supercomputers, but this investment is very huge. The second method is to optimize the data organization, the most common method is to use multi-level resolution technology. This technology is widely used in the field of geographic information system and remote sensing images.

In the Chinese invention patent specification CN102663140A, a fast access method for TB-level images is proposed. This method is realized by establishing image indexes and image blocks. In the way of graph, the memory buffer pool and video memory buffer pool technologies are used to realize fast data access. However, the images processed by this method are only two-dimensional images, and the index established is suitable for specific application fields. Moreover, index files are created for each small file, and the number is huge, which consumes computing resources.

In the Chinese invention patent specification CN103440350A, a three-dimensional data retrieval method based on an octree is proposed. The method establishes an octree file for the three-dimensional data and establishes a data block index table corresponding to each octree node. By querying the data Block corresponding identification, to find the corresponding data block. Complete a retrieval request for 3D data. The minimum unit of the three-dimensional data block retrieved each time by this method is the size of each octree data block.

The above two patents are mainly for the massive data generated in geographic remote sensing. Compared with remote sensing images, three-dimensional brain tissue images contain a lot of organizational structure information, such as blood vessels and neural networks, with complex structures. Researchers use coordinates to represent the positions of certain structures in the brain tissue. When calling the region of interest, it also uses its coordinate information in the entire image to complete the call, and it is not suitable to use index files. On the other hand, due to the characteristics of multiple structures in the brain tissue, it has a certain continuity in the three-dimensional space, so these continuous images cannot be stored in a two-dimensional manner.

Some commercial software widely used in brain science research has also proposed solutions for archiving 3D massive data, such as the Large Data Access (LDA) module in Amira software and the Large Data Management (LDM) development kit in the OpenInventor visualization development platform. Whether it is LDA or LDM, all the data (different resolution levels) are archived in a single file, which is not suitable for the storage and transmission of more than hundreds of GB of data, and it is not easy to achieve parallelization when data is archived.

Contents of the invention

The technical problem to be solved by the present invention is to provide a rapid extraction method for high-resolution mass atlases of three-dimensional brain tissue, to archive high-resolution mass atlases of three-dimensional brain tissue, and to quickly and continuously retrieve Interested data blocks can greatly improve the computer system's ability to process three-dimensional massive image data sets obtained in brain science research without increasing hardware investment.

A data rapid archiving method suitable for high-resolution massive atlases of three-dimensional brain tissue, characterized in that it includes the following steps:

1) Generate archived data:

Divide the three-dimensional data space into a data set D0 composed of several data blocks of equal size, each data block has a size of x, y, z;

The three-dimensional data space is subjected to equal-scale three-dimensional sampling, and divided into low-resolution data sets D1 composed of several equal-sized data blocks, and the size of each data block is also x, y, and z;

By analogy, carry out n equal-scale three-dimensional sampling, and obtain n+1 sets of data sets D0, D1, D2, D3...Dn with different resolution levels, and the size of the data blocks that make up each data set are x, y ,z;

Save all data blocks as independent data files, and store them according to the path self-index structure;

Create a parameter file, which records the voxel resolution of the original data, the size of each data block, and the multiple of each sampling;

2) Extract archived data:

The information provided by the parameter file includes the voxel resolution, data block size, sampling multiple, the starting and ending coordinates of X, Y, and Z of the required region of interest, and the number of required resolution levels. These six groups of parameters , calculating the minimum data blocks required to contain the region of interest;

sequentially read into memory the independent data files of the data blocks required for the region of interest, but retain only the data points for the region of interest;

All data points held in memory and the individual data files of the data blocks read into memory form the data space of interest.

The format adopted by the independent data file described in step 1) is a three-dimensional image storage format.

The self-index structure described in step 1) is a four-layer path structure, and each layer is defined as:

The first level is the root directory;

The second layer is the resolution level, which is distinguished by folder name;

The third layer is the Z direction, that is, the block index number in the axial direction of the original 3D atlas, which is distinguished by the folder name;

The fourth layer is the block index number in the X or Y direction, which is distinguished by the folder name. Each folder contains the data file corresponding to the relevant data block, and the file name includes the block index number of X, Y, and Z.

The main features of the present invention are: the high-resolution mass atlas of three-dimensional brain tissue can be archived, and interested data blocks can be quickly and continuously retrieved from the archived file without an index file. Under the condition that the storage capacity allows, the ordinary desktop computer can realize the processing capacity of at least 10 TB massive data by applying the present invention. In addition, the process of creating and extracting documents in the present invention has parallelism, which can be further accelerated by applying parallel computing.

Description of drawings

Fig. 1 is a schematic diagram of generating archived data in the present invention;

Fig. 2 is a schematic diagram of the self-index organization form of the three-dimensional small block of the present invention;

Fig. 3 is a schematic flowchart of calling the three-dimensional region of interest in the present invention.

Detailed ways

The present invention is a fast data extraction method applicable to high-resolution mass atlases of three-dimensional brain tissue, which divides two-dimensional continuous sequence images into three-dimensional small blocks at multiple resolutions, and establishes a folder organization form according to a self-indexing method And the file name naming method, by providing the coordinate information and resolution level of the 3D block to be called, call the 3D small block and calculate the 3D block in the area. In this way, the data archiving and calling of the high-resolution massive atlas of three-dimensional brain tissue can be completed.

The following structural drawings and embodiments describe the present invention in further detail.

As shown in Figure 1, the data archiving method for the high-resolution massive atlas of three-dimensional brain tissue provided by this embodiment may include the following steps:

1. Generate archived data.

A two-dimensional graphics sequence can be viewed as a three-dimensional image stack. First, the entire 3D image stack is split into multiple 3D data blocks (referred to as 3D small blocks) with a size of 512×512×512 pixels to form a data set D0; Sampling is doubled, and one image is extracted for every two images, and then the new image sequence is split to obtain multiple three-dimensional small blocks with a size of 512×512×512 pixels to form a data set D1; and so on until X The size of the three directions of , Y, and Z is sampled to be less than or equal to 512 pixels, forming a total of n+1 sets of data sets with different resolutions in D0, D1...Dn. The generated schematic diagram can refer to Figure 1.

The three-dimensional small blocks with a size of 512×512×512 split above are saved in a three-dimensional TIFF format, but may also be in a three-dimensional matrix format such as RAW, or even a video format. When calling data, different calling methods are used for different formats, which ensures the flexibility and expansibility of this method.

All three-dimensional small blocks are named and stored according to a certain folder organization method and file naming method, and parameter files are established to form a self-index organization. The form of folder and file organization can be shown in Figure 2:

The first is the root directory, which contains a description file, which is used to describe the overall information of the 3D image stack, including the size of the 3D image stack in the XYZ direction, the resolution of the three directions of the image, and how many different resolutions are formed in total Level dataset, how many blocks are split in each direction for each resolution level, and the total folder path where the split files are saved.

There are also P subfolders under the root directory, representing different resolution series, namely D0, D1, D2...Dn, a total of n+1 sets. The folder prefix is "level". Under each resolution series folder, it contains Q Z-direction block number folders. The folder prefix is "z", and Q is the number of blocks in the Z direction at the current resolution level. . Under each block number folder in the Z direction, there are R block number folders in the Y direction, the folder prefix is "y", and R is the number of blocks in the Y direction at this resolution. Under each Y-direction block number folder, there are S three-dimensional small blocks, and S is the number of blocks in the X-direction at this resolution.

The file name of each 3D small block is named in the form of X_Y_Z, where X represents the sequence block number of the entire 3D image in the x direction at the resolution level of the 3D small block, and Y represents the resolution level of the 3D small block at the current resolution level , the sequence block number in the Y direction of the entire 3D image, Z represents the sequence block number in the Z direction of the entire 3D image at the resolution level of the 3D small block, and the value of X is P/512, where P is the entire 3D image in the Under the resolution level, the size of the image in the X direction can be calculated in the same way as Y and Z.

For example, the original data with a pixel size of 2000×1500×1000 has a resolution of 1×1×1, which can be split into three sets of data sets D0, D1, and D2 in total, so the number of resolution folders P is 3 , the folders are "level0", "level1", and "level2". The number of blocks in the Z direction in the D0 data set is 1000/512+1=2, the number of blocks in the Y direction in the D0 data set is 1500/512+1=3, and the number of blocks in the X direction in the D0 data set is 2000/512+1= 4, so the "level0" folder contains two folders "z0" and "z1", and each Z-direction block number folder contains three folders "y0", "y1" and "y2" , containing 4 small three-dimensional blocks in each y-direction block number folder. The four three-dimensional small blocks under the level0/z0/y0 folder are named 0_0_0, 1_0_0, 2_0_0, 3_0_0, and the four three-dimensional small blocks under the level0/z0/y1 folder are named 0_1_0, 1_1_0, 2_1_0, 3_1_0, in the same way, you can get the file names of the 3D small blocks in other folders. D1 and D2 dataset folder structure and so on.

2. Extract archived data.

The user proposes the coordinate range of the region of interest that needs to be called, including the start and end coordinates of X, Y, Z and the 7 parameters of the resolution level. Through calculation, the start and end coordinates of X, Y, and Z can be obtained The range of block index numbers in three directions of the three-dimensional small block corresponding to the coordinates.

Obtain the 3D image of the desired region of interest through the second call.

For example, when the user needs to call a region of interest whose starting coordinates are 201, 301, 0, ending coordinates are 800, 900, 399, and the resolution is 0-level 600×600×400, by calculation, X The starting block number in the direction is 201/512=0, the ending block number is 800/512=1, the starting block number in the Y direction is 301/512=0, the ending block number is 900/512=1, starting from the Z direction The starting block number is

0/512=0, the end block number is 399/512=0. So a total of 2×2×1=4 blocks need to be read, and the path to read is:

Data_floder/level0/z0/y0/0_0_0

Data_floder/level0/z0/y0/1_0_0

Data_floder/level0/z0/y1/0_1_0

Data_floder/level0/z0/y1/1_1_0

This step completes the process of fetching the 3D patch into memory.

After reading all the required three-dimensional small blocks into the memory, these three-dimensional small blocks contain the regions of interest required by the user. Through the fine calculation process, the three-dimensional small blocks will be required to combine the regions of interest into a complete three-dimensional Data space, complete the calling process. As shown in Figure 3.

Claims (3)

1. be applicable to the quick archiving method of data of three-dimensional brain tissue high-resolution magnanimity atlas, it is characterized in that comprising the following steps:

1) filing data is generated:

Be by some equal and opposite in direction data blocks by three-dimensional data compartition, the data set D0 formed, each data block is of a size of x, y, z;

Equal proportion three-dimensional sample is carried out in described three-dimensional data space, is divided into the low-resolution data collection D1 be made up of some equal and opposite in direction data blocks, the size of each data block is also x, y, z;

By parity of reasoning, carries out n equal proportion three-dimensional sample, obtains data set D0, D1, D2, D3 that n+1 cover has different resolution level ... Dn, the size forming the data block of each data set is x, y, z;

All data blocks are saved as independent data document, and stores from index structure according to path;

Set up Parameter File, Parameter File record raw data voxel resolution, the size of every blocks of data block, the multiple of each sampling;

2) filing data is extracted:

The information of the voxel resolution provided by Parameter File, block size, sampling multiple, the origin coordinates of X, Y, Z of required area-of-interest and termination coordinate, and these 6 groups of parameters of required level of resolution number, calculate the minimal data block comprised needed for described area-of-interest;

Successively the independent data document of the data block needed for area-of-interest is read in internal memory, but only retain the data point of area-of-interest;

All data points be retained in internal memory form interested data space with the independent data document of the data block of reading in internal memory.

2., according to the quick archiving method of data being applicable to three-dimensional brain tissue high-resolution magnanimity atlas described in claim 1, it is characterized in that, step 1) described in the form that adopts of independently data file be 3-D view storage format.

3., according to the quick archiving method of data being applicable to three-dimensional brain tissue high-resolution magnanimity atlas described in claim 1, it is characterized in that, step 1) described in be a kind of four layers of path structure from index structure, every one deck is defined as:

Ground floor is root directory;

The second layer is level of resolution, distinguishes with Folder Name;

Third layer is Z-direction, and namely the block call number of former three-dimensional atlas axis, distinguishes with Folder Name;

4th layer is the block call number of X or Y-direction, distinguishes, comprise the data file corresponding to relevant data block, comprise the block index number of X, Y and Z in document name in each file with Folder Name.