CN104615739A - Rapid data archiving method applicable to three-dimensional brain tissue high-resolution mass atlases - Google Patents
Rapid data archiving method applicable to three-dimensional brain tissue high-resolution mass atlases Download PDFInfo
- Publication number
- CN104615739A CN104615739A CN201510072037.XA CN201510072037A CN104615739A CN 104615739 A CN104615739 A CN 104615739A CN 201510072037 A CN201510072037 A CN 201510072037A CN 104615739 A CN104615739 A CN 104615739A
- Authority
- CN
- China
- Prior art keywords
- data
- dimensional
- resolution
- block
- interest
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1858—Parallel file systems, i.e. file systems supporting multiple processors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/113—Details of archiving
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/56—Information retrieval; Database structures therefor; File system structures therefor of still image data having vectorial format
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention relates to a rapid data archiving method applicable to three-dimensional brain tissue high-resolution mass atlases. The rapid data archiving method includes the steps of generating archiving data; partitioning the three-dimensional data space into a plurality of data blocks identical in size and generating n+1 data sets with n+1 sorts of different resolution, wherein the data sets are stored as independent data documents according to path self-index structures; extracting the archiving data, namely calculating the minimum number of data blocks needed in area-of-interest through six parameters of starting coordinates, terminating coordinates and high resolution of the area-of-interest; sequentially reading the independent data documents of the data blocks needed in the area-of-interest into memory and allowing the independent data documents to form the interesting data space with data points keeping the area-of-interest. The three-dimensional brain tissue high-resolution mass atlases are archived, and the interesting data blocks can be extracted rapidly and continuously from the archived documents without index files. Parallelism in establishing and extracting the documents is achieved, and further acceleration can be realized by applying parallel computing.
Description
Technical field
The present invention is applicable to Biomedical Image process field, more specifically, be applicable to the filing of three-dimensional brain tissue high-resolution magnanimity atlas, and data of interest block can be transferred out rapidly, continuously from archive file, greatly improve when not increasing hardware and dropping into computer system in brain science research obtain the processing power of three-dimensional mass image data collection, be specifically related to a kind of quick archiving method of data being applicable to three-dimensional brain tissue high-resolution magnanimity atlas.
Background technology
In present brain science research, along with the development of imaging technique, researchist can carry out imaging to large-scale brain tissue on the one hand, on the other hand, imaging can be carried out to meticulousr brain tissue, along with the increase of areas imaging and imaging precision, the image data amount produced is also very huge, often reaches TB level.How processing these magnanimity Brain Tissues Image data is very large challenges.Usual employing two kinds of solutions.The first promotes level of hardware exactly, such as supercomputer, but this input is very huge.Second method is optimized Organization of Data exactly, and the most frequently used method adopts multiresolution technology exactly.It is more that this technology is used in Geographic Information System and remote sensing images field.
A kind of quick access method for TB level image is proposed in Chinese invention patent instructions CN102663140A, the method is by setting up image index and image block realization, according to the scene that image is applied, set up data directory, and image is converted to the mode of little figure, utilize memory buffer pond and video memory Buffer Pool technology to realize data fast access.But the image handled by the method is only two dimensional image, the index set up is applicable to specific application.And index file sets up for each small documents, enormous amount, consumption calculations resource.
In Chinese invention patent instructions CN103440350A, propose a kind of three-dimensional data search method based on Octree, the method is by setting up Octree file by three-dimensional data and setting up corresponding each octree nodes data block concordance list, by the mark that data query block is corresponding, find corresponding data block.Complete the retrieval request of three-dimensional data.The least unit of the three-dimensional data block that the method is retrieved at every turn is the size of each Octree data block.
The mass data that above-mentioned two patents mainly produce in geographical remote sensing, compared with remote sensing images, comprises many organizational information in three-dimensional Brain Tissues Image, such as blood vessel, neural network, complex structure.Researchist is adopt coordinate to represent to the position of some structure in brain tissue, is also transferred by its coordinate information in whole image, is not suitable for adopting index file when transferring area-of-interest.On the other hand due to structures many in brain tissue, it has certain continuity in three dimensions, so the mode of two dimension can not be adopted to store these consecutive images.
In brain science research, more widely used business softwares it is also proposed the solution of filing three-dimensional mass data, such as, Large Data Access (LDA) module in Amira software and Large Data Management (LDM) kit in OpenInventor visual development platform.No matter be LDA or LDM, all data (different resolution level) be archived in a single document, and be not suitable for storage and the transmission of hundreds of more than GB data, during data filing, also not easily realize parallelization.
Summary of the invention
Technical matters to be solved by this invention is to provide a kind of rapid extracting method for three-dimensional brain tissue high-resolution magnanimity atlas, three-dimensional brain tissue high-resolution magnanimity atlas is filed, and data of interest block can be transferred out rapidly, continuously from archive file, greatly improve when not increasing hardware and dropping into computer system in brain science research obtain the processing power of three-dimensional mass image data collection.
Be applicable to the quick archiving method of data of three-dimensional brain tissue high-resolution magnanimity atlas, it is characterized in that comprising the following steps:
1) filing data is generated:
Be by some equal and opposite in direction data blocks by three-dimensional data compartition, the data set D0 formed, each data block is of a size of x, y, z;
Equal proportion three-dimensional sample is carried out in described three-dimensional data space, is divided into the low-resolution data collection D1 be made up of some equal and opposite in direction data blocks, the size of each data block is also x, y, z;
By parity of reasoning, carries out n equal proportion three-dimensional sample, obtains data set D0, D1, D2, D3 that n+1 cover has different resolution level ... Dn, the size forming the data block of each data set is x, y, z;
All data blocks are saved as independent data document, and stores from index structure according to path;
Set up Parameter File, Parameter File record raw data voxel resolution, the size of every blocks of data block, the multiple of each sampling;
2) filing data is extracted:
The information of the voxel resolution provided by Parameter File, block size, sampling multiple, the origin coordinates of X, Y, Z of required area-of-interest and termination coordinate, and these 6 groups of parameters of required level of resolution number, calculate the minimal data block comprised needed for described area-of-interest;
Successively the independent data document of the data block needed for area-of-interest is read in internal memory, but only retain the data point of area-of-interest;
All data points be retained in internal memory form interested data space with the independent data document of the data block of reading in internal memory.
Step 1) described in the form that adopts of independently data file be 3-D view storage format.
Step 1) described in be a kind of four layers of path structure from index structure, every one deck is defined as:
Ground floor is root directory;
The second layer is level of resolution, distinguishes with Folder Name;
Third layer is Z-direction, and namely the block call number of former three-dimensional atlas axis, distinguishes with Folder Name;
4th layer is the block call number of X or Y-direction, distinguishes, comprise the data file corresponding to relevant data block, comprise the block index number of X, Y and Z in document name in each file with Folder Name.
Principal feature of the present invention is: can file three-dimensional brain tissue high-resolution magnanimity atlas, just can transfer out data of interest block rapidly, continuously from archive file without the need to index file.When storage capacity allows, common desktop computer application the present invention, at least can realize the processing power of 10TB mass data.In addition, the process setting up and extract document in the present invention has concurrency, can do to accelerate further by application parallel computation.
Accompanying drawing explanation
Fig. 1 is that the present invention generates filing data schematic diagram;
Fig. 2 be the three-dimensional fritter of the present invention from index organization's form schematic diagram;
Fig. 3 is the schematic flow diagram of the present invention when calling 3D region interested.
Embodiment
A kind of Fast Data Extraction Methodology being applicable to three-dimensional brain tissue high-resolution magnanimity atlas of the present invention, by carrying out to two-dimentional continuous sequence figure the mode that multiresolution is split as three-dimensional fritter, and according to setting up folder organization form and filename naming method from indexed mode, by providing coordinate information and the level of resolution of the three-dimensional bits that will call, calling three-dimensional fritter and calculating this area three-dimensional block.Thus complete the data filing of three-dimensional brain tissue high-resolution magnanimity atlas and call.
The present invention is described in further detail for Structure Figure and embodiment below.
As shown in Figure 1, the data archiving method of the three-dimensional brain tissue high-resolution magnanimity atlas that the present embodiment provides, can comprise the following steps:
1. generate filing data.
X-Y scheme sequence can regard a three-dimensional image storehouse as.First whole 3-D view storehouse is split as the three-dimensional data block (being called for short three-dimensional fritter) that multiple size is 512 × 512 × 512 pixels, forms data set D0; Then original two-dimensional image sequence is often opened X, Y-direction samples 2 times, every two image contracts one, and then split new image sequence, obtain the three-dimensional fritter that multiple size is 512 × 512 × 512 pixels, form data set D1; By that analogy, until the size in X, Y, Z tri-directions samples all be less than or equal to 512 pixels, form D0, D1 ... Dn altogether n+1 overlaps the data set of different resolution.The schematic diagram generated can reference diagram 1.
The three-dimensional fritter of above-mentioned 512 × 512 × 512 sizes split into, saves as three-dimensional tiff format, but also can be the three-dimensional matrice forms such as RAW, or even video format.Use different call methods when calling data for different forms, ensure that dirigibility and the extendability of this method.
Carry out name according to certain folder organization mode and file designation mode to all three-dimensional fritters to store, and set up Parameter File, formed from index organization.The form of file and file organization can as shown in Figure 2:
First be root directory, a description document is comprised under root directory, this description document is for describing the overall information of 3-D view storehouse, comprise the size in 3-D view storehouse XYZ direction, the resolution in image three directions, form altogether the data set of how many different resolution grades, each direction of each level of resolution has split how many blocks respectively, and the general act folder path that after splitting, file is preserved.
Also comprise P sub-folder under the root directory, represent different resolution progression, i.e. D0, D1, D2 ... Dn, altogether n+1 cover.File prefix is " level ", and under each resolution progression file, comprise Q Z-direction block file, file prefix is " z ", and Q is the block number of Z-direction under this level of resolution current.Under each Z-direction block file, comprise R Y-direction block file, file prefix is " y ", and R is the block number of Y-direction under this resolution.Under each Y-direction block file, comprise S three-dimensional fritter, S is the block number of X-direction under this resolution.
The file designation of each three-dimensional fritter is named in the mode of X_Y_Z, wherein X represents that this three-dimensional fritter is under the level of resolution of place, the sequence blocks number in whole 3-D view x direction, Y represents that this three-dimensional fritter is under the level of resolution of place, in the sequence blocks number of whole 3-D view Y-direction, Z represents that this three-dimensional fritter is under the level of resolution of place, in the sequence blocks number of whole 3-D view Z-direction, the value of X is P/512, wherein P is that whole three-dimensional is under the level of resolution of place, size in image X-direction, in like manner can obtain the account form of Y and Z.
Such as a pixel size is the raw data of 2000 × 1500 × 1000, its resolution is 1 × 1 × 1, altogether can split into D0, D1, D2 tri-sets of data collection, so level of resolution file number P is 3, file is respectively " level0 ", " level1 ", " level2 ".Be 1000/512+1=2 at the block number of D0 data centralization Z-direction, the block number 1500/512+1=3 of D0 data centralization Y-direction is individual, the block number 2000/512+1=4 of D0 data centralization X-direction is individual, so comprise " z0 ", " z1 " 2 files in " level0 " file, in each Z-direction block file, comprise " y0 ", " y1 ", " y2 " three files, in the block file of each y direction, comprise 4 three-dimensional fritters.4 three-dimensional fritters under level0/z0/y0 file, title is respectively 0_0_0,1_0_0,2_0_0,3_0_0,4 three-dimensional fritters under level0/z0/y1 file, title is respectively 0_1_0,1_1_0,2_1_0,3_1_0, in like manner can obtain the filename of the lower three-dimensional fritter of alternative document folder.D1 and D2 document data set clamping structure by that analogy.
2. extract filing data.
User proposes the coordinate range of the area-of-interest needing to call, comprise the origin coordinates of X, Y, Z and stop coordinate and which class resolution ratio grade 7 parameters, by calculating the block call number scope that can obtain X, Y, Z origin coordinates and stop three-dimensional fritter three directions corresponding to coordinate.
The 3-D view obtaining required area-of-interest is called by secondary.
Illustrate, be 201,301,0 when user needs to transfer origin coordinates, stopping coordinate is 800,900,399, when resolution is the area-of-interest of 600 × 600 × 400 sizes of the 0th grade, by calculating, the initial block number of X-direction is 201/512=0, stopping block number is 800/512=1, the initial block number of Y-direction is 301/512=0, and stopping block number is 900/512=1, and the initial block number of Z-direction is
0/512=0, stopping block number is 399/512=0.So altogether need reading 2 × 2 × 1=4 block, the path that read is:
Data_floder/level0/z0/y0/0_0_0
Data_floder/level0/z0/y0/1_0_0
Data_floder/level0/z0/y1/0_1_0
Data_floder/level0/z0/y1/1_1_0
This step completes the process extracted by three-dimensional fritter in internal memory.
Three-dimensional fritter in need for institute is read after in internal memory, the area-of-interest required for user is contained in these three-dimensional fritters, by Exact calculation process, area-of-interest is combined into a complete three-dimensional data space, complete call process by needing with three-dimensional fritter.As shown in Figure 3.
Claims (3)
1. be applicable to the quick archiving method of data of three-dimensional brain tissue high-resolution magnanimity atlas, it is characterized in that comprising the following steps:
1) filing data is generated:
Be by some equal and opposite in direction data blocks by three-dimensional data compartition, the data set D0 formed, each data block is of a size of x, y, z;
Equal proportion three-dimensional sample is carried out in described three-dimensional data space, is divided into the low-resolution data collection D1 be made up of some equal and opposite in direction data blocks, the size of each data block is also x, y, z;
By parity of reasoning, carries out n equal proportion three-dimensional sample, obtains data set D0, D1, D2, D3 that n+1 cover has different resolution level ... Dn, the size forming the data block of each data set is x, y, z;
All data blocks are saved as independent data document, and stores from index structure according to path;
Set up Parameter File, Parameter File record raw data voxel resolution, the size of every blocks of data block, the multiple of each sampling;
2) filing data is extracted:
The information of the voxel resolution provided by Parameter File, block size, sampling multiple, the origin coordinates of X, Y, Z of required area-of-interest and termination coordinate, and these 6 groups of parameters of required level of resolution number, calculate the minimal data block comprised needed for described area-of-interest;
Successively the independent data document of the data block needed for area-of-interest is read in internal memory, but only retain the data point of area-of-interest;
All data points be retained in internal memory form interested data space with the independent data document of the data block of reading in internal memory.
2., according to the quick archiving method of data being applicable to three-dimensional brain tissue high-resolution magnanimity atlas described in claim 1, it is characterized in that, step 1) described in the form that adopts of independently data file be 3-D view storage format.
3., according to the quick archiving method of data being applicable to three-dimensional brain tissue high-resolution magnanimity atlas described in claim 1, it is characterized in that, step 1) described in be a kind of four layers of path structure from index structure, every one deck is defined as:
Ground floor is root directory;
The second layer is level of resolution, distinguishes with Folder Name;
Third layer is Z-direction, and namely the block call number of former three-dimensional atlas axis, distinguishes with Folder Name;
4th layer is the block call number of X or Y-direction, distinguishes, comprise the data file corresponding to relevant data block, comprise the block index number of X, Y and Z in document name in each file with Folder Name.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510072037.XA CN104615739B (en) | 2015-02-11 | 2015-02-11 | The quick archiving method of data suitable for three-dimensional brain tissue high-resolution magnanimity atlas |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510072037.XA CN104615739B (en) | 2015-02-11 | 2015-02-11 | The quick archiving method of data suitable for three-dimensional brain tissue high-resolution magnanimity atlas |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104615739A true CN104615739A (en) | 2015-05-13 |
| CN104615739B CN104615739B (en) | 2018-10-09 |
Family
ID=53150181
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510072037.XA Active CN104615739B (en) | 2015-02-11 | 2015-02-11 | The quick archiving method of data suitable for three-dimensional brain tissue high-resolution magnanimity atlas |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104615739B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108513136A (en) * | 2018-04-08 | 2018-09-07 | 华中科技大学苏州脑空间信息研究院 | The compression method and read method of magnanimity space three-dimensional image data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1602059A (en) * | 2003-09-28 | 2005-03-30 | 乐金电子（惠州）有限公司 | Audio file selective playing method having catalogue structure |
| CN101388043A (en) * | 2008-09-26 | 2009-03-18 | 北京航空航天大学 | OGC high performance remote sensing image map service method based on small picture |
| CN101719148A (en) * | 2009-11-24 | 2010-06-02 | 北京灵图软件技术有限公司 | Three-dimensional spatial information saving method, device, system and dispatching system |
| CN101887595A (en) * | 2009-05-14 | 2010-11-17 | 武汉如临其境科技创意有限公司 | Three-dimensional digital earth-space data organizing and rendering method based on quad-tree index |
-
2015
- 2015-02-11 CN CN201510072037.XA patent/CN104615739B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1602059A (en) * | 2003-09-28 | 2005-03-30 | 乐金电子（惠州）有限公司 | Audio file selective playing method having catalogue structure |
| CN101388043A (en) * | 2008-09-26 | 2009-03-18 | 北京航空航天大学 | OGC high performance remote sensing image map service method based on small picture |
| CN101887595A (en) * | 2009-05-14 | 2010-11-17 | 武汉如临其境科技创意有限公司 | Three-dimensional digital earth-space data organizing and rendering method based on quad-tree index |
| CN101719148A (en) * | 2009-11-24 | 2010-06-02 | 北京灵图软件技术有限公司 | Three-dimensional spatial information saving method, device, system and dispatching system |
Non-Patent Citations (1)
| Title |
|---|
| 闵阳: "基于多源空间数据的三维可视化", 《中国优秀硕士学位论文全文数据库 基础科学辑》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108513136A (en) * | 2018-04-08 | 2018-09-07 | 华中科技大学苏州脑空间信息研究院 | The compression method and read method of magnanimity space three-dimensional image data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104615739B (en) | 2018-10-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111291016B (en) | Hierarchical hybrid storage and indexing method for massive remote sensing image data | |
| CN107133325B (en) | Internet photo geographic space positioning method based on street view map | |
| Roosevelt et al. | Excavation is destruction digitization: advances in archaeological practice | |
| CN103595791B (en) | Cloud accessing method for mass remote sensing data | |
| Burns et al. | The open connectome project data cluster: scalable analysis and vision for high-throughput neuroscience | |
| CN104077411B (en) | remote sensing satellite data processing method and system | |
| CN110781325A (en) | High-resolution remote sensing data grid refined management model and construction method thereof | |
| CN105786942A (en) | Geographic information storage system based on cloud platform | |
| CN104820714A (en) | Mass small tile file storage management method based on hadoop | |
| CN108804602A (en) | A kind of distributed spatial data storage computational methods based on SPARK | |
| CN103020325A (en) | Distributed remote sensing data organization query method based on NoSQL database | |
| Xiao et al. | Remote sensing image database based on NOSQL database | |
| Jones et al. | High‐resolution digital brain atlases: a Hubble telescope for the brain | |
| CN104134214A (en) | Digital slice image storing method and digital slice image displaying method based on image pyramid layering | |
| CN110659369B (en) | On-orbit high-precision lightweight global image control point database construction method and system | |
| Eldawy et al. | Large scale analytics of vector+ raster big spatial data | |
| Adorjan | Opensfm: A collaborative structure-from-motion system | |
| Jin et al. | Pai-wsit: An AI service platform with support for storing and sharing whole-slide images with metadata and annotations | |
| CN114090800A (en) | Multi-source heterogeneous data aggregation system of urban information model | |
| CN104615739A (en) | Rapid data archiving method applicable to three-dimensional brain tissue high-resolution mass atlases | |
| CN111061806B (en) | Storage method and networked access method for distributed massive geographic tiles | |
| Li et al. | An improved distributed storage model of remote sensing images based on the HDFS and pyramid structure | |
| Yang et al. | Mobile image search with multimodal context-aware queries | |
| CN114283258A (en) | CNN-based method for generating three-dimensional point cloud from single image | |
| Rajkumar et al. | A review of visual information retrieval on massive image data using hadoop |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |