CN103886045A - Access method and device including local valid grid data - Google Patents
Access method and device including local valid grid data Download PDFInfo
- Publication number
- CN103886045A CN103886045A CN201410088639.XA CN201410088639A CN103886045A CN 103886045 A CN103886045 A CN 103886045A CN 201410088639 A CN201410088639 A CN 201410088639A CN 103886045 A CN103886045 A CN 103886045A
- Authority
- CN
- China
- Prior art keywords
- raster data
- pixel
- data
- grid
- mark
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
Abstract
The invention provides an access method and device including local valid grid data. The method comprises the steps that valid grid pixels and invalid grid pixels in the grid data are marked to generate marked grid data; the marked grid data are compressed to obtain compressed and marked grid data; pixel values of the valid grid pixels in the grid data are independently stored to obtain pixel value data strings of the valid grid pixels, and basic information of the grid data is recorded; when an application system needs to store the grid data, the compressed and marked grid data, the pixel value data strings of the valid grid pixels and the basic information of the grid data are used as storage objects to be written in files to be stored. According to the method and device, the storage amount of the grid data can be greatly reduced, and storage space can be saved; the processed grid data are transmitted, the transmission amount of the gird data can be greatly reduced, and network grid data transmission efficiency can be improved.
Description
Technical field
The present invention relates to area of geographic information, particularly relate to a kind of comprise local effectively access method and the device of raster data.
Background technology
Raster data is at GIS (Geographic Information System, Geographic Information System) in be widely used, in actual items, one of its important feature is that data volume is large, and the transmission quantity of the raster data between server and client is often the key factor of restriction gis application system performance.In raster data transmitting procedure, general way and the method for current main-stream are all the data objects take all pixel values of the rectangular area of a certain pixel width and a certain pixel height composition as Internet Transmission.This method is less (such as rectangular area pixel summation is less than 10000 to number of total picture element amount in rectangular area, every pixel data needs 4 bytes of storage space, the total amount of data of transmission is less than 40Kb so) in situation, in most Gis application, be acceptable; But, when the rectangular area that needs transmission in application comprises pixel number when larger, such as the pixel area of 1024*1024, every pixel data needs 4 bytes of storage space, so total transmission data 4096Kb is just very large, and the impact of the performance on application system is just very large.
At present, to how reducing the transmission quantity of raster data, improving system effectiveness has more research, but mainly concentrates on compression to data or the technique direction of coding, obtains minimum memory space and transmission quantity.
Summary of the invention
The invention provides a kind of access method and device that comprises local effective raster data in GIS application, the target problem solving has distribution randomness, quantity and has the invalid grid pixel collection and local effectively grid pixel collection differentiating and processing of randomness from raster data, can greatly reduce the memory space of raster data, save storage space; Raster data after treatment is transmitted, can greatly reduce the transmission quantity of grid number, improve network raster data transfer efficiency.
For solving the problems of the technologies described above, the invention provides a kind of local effectively access method of raster data that comprises,
In raster data, comprise effective grid pixel and invalid grid pixel, the value of invalid grid pixel is data manufacturer's designated value;
Effective grid pixel in raster data and invalid grid pixel are carried out to mark, generate mark raster data; Mark raster data is compressed, obtain compact token raster data;
The pixel value of the effective grid pixel in raster data is stored separately in certain sequence, abandoned the processing of invalid grid pixel partial data, obtain the pixel value serial data of effective grid pixel;
Record the essential information of raster data, essential information comprises value and the pixel data type of pixel width, pixel height, invalid grid pixel;
In the time that application system need to be stored raster data, using the essential information of the pixel value serial data of described compact token raster data, effective grid pixel, raster data as storage object, writing in files is stored;
In the time that application system need read the raster data of storage, the essential information of the raster data in first file reading, and according to the essential information of raster data, distribute the two-dimensional matrix storage space with the corresponding size of described raster data, according to the pixel value serial data of compact token raster data and effective grid pixel, two-dimensional matrix is processed again, obtained the raster data restoring.
Processing to raster data, is all pixels based in raster data at present, and then to these pixel datas, how research is compressed, and coding etc., obtain minimum memory space and transmission quantity; This law is bright is first from raster data, to distinguish Useful Information, abandons useless information, so just can reduce data volume to a great extent, certainly applies the processing that can continue then to carry out data compression aspect after this method.
On above-mentioned a kind of basis that comprises the local effectively access method technical scheme of raster data, the present invention can also do following improvement.
Further, described effective grid pixel in raster data and invalid grid pixel are carried out to mark, generate mark raster data; Mark raster data is compressed, and the step that obtains compact token raster data comprises:
Whether the value that judges successively each pixel in described raster data is the value of invalid grid pixel;
If so, this grid pixel is invalid grid pixel, uses 0 mark;
If not, this grid pixel is effective grid pixel, uses 1 mark;
Record in certain sequence the mark of all pixels in described raster data, form mark raster data;
Described mark raster data is accounted for to 1bit storage space according to every pixel to be redistributed;
Be one group by the mark raster data after described redistributing according to starting continuous 32 from the initial number of mark raster data, and by 32 every group the flag data of 4 bytes get the mode that its decimal integer value stores and compress, obtain compact token raster data.
The beneficial effect of technique scheme is: the label information disposal route that the present invention adopts, no matter how can realize the distribution of invalid grid pixel, quantity how, the duplicate target of compression efficiency, can meet raster data in actual production and often have the situation without data value of a large amount of dispersivenesses.
Further, described according to the essential information of raster data, the two-dimensional matrix storage space of distribution and the corresponding size of described raster data, then according to the pixel value serial data of compact token raster data and effective grid pixel, two-dimensional matrix is processed, the step that obtains the raster data restoring comprises:
According to the essential information of described raster data, distribute the two-dimensional matrix storage space with the corresponding size of described raster data; Each grid of this two-dimensional matrix of initialization is " untreated state ";
The described compact token raster data that decompresses, gives described two-dimensional matrix by described mark raster data, marks invalid grid pixel and the local effectively position of grid pixel in described two-dimensional matrix;
The relevant position of the invalid grid pixel of described two-dimensional matrix is inserted to the value of invalid grid pixel, and grid state is set to " processed state ";
Again the pixel value serial data of described effective grid pixel is inserted in certain sequence to the grid of " untreated state " in described two-dimensional matrix, obtained the raster data restoring at receiving end.
The invention also discloses a kind of local effectively access device of raster data that comprises, comprising:
Raster data module, for storing original raster data, comprises in described raster data in raster data and comprises effective grid pixel and invalid grid pixel, and the value of invalid grid pixel is data manufacturer's designated value;
The mark module of the local valid pixel collection of raster data, for effective grid pixel of described raster data and invalid grid pixel are carried out to mark, generates mark raster data; Described mark raster data is compressed, obtain compact token raster data;
The actual pixel value module of local valid pixel collection, for the pixel value of effective grid pixel of described raster data is stored separately in certain sequence, abandons the processing of invalid grid pixel partial data, obtains the pixel value serial data of effective grid pixel;
The basic information module of raster data, for recording the essential information of described raster data, essential information comprises value and the pixel data type of pixel width, pixel height, invalid grid pixel;
Memory module, in the time that application system need to be stored raster data, using the essential information of the pixel value serial data of described compact token raster data, effective grid pixel, raster data as storage object, writing in files is stored;
Read module, for in the time that application system need to read the raster data of storage, the essential information of the raster data in first file reading, and according to the essential information of raster data, distribute the two-dimensional matrix storage space with the corresponding size of described raster data, according to the pixel value serial data of compact token raster data and effective grid pixel, two-dimensional matrix is processed again, obtained the raster data restoring.
On above-mentioned a kind of access device basis that comprises local effectively raster data, the present invention can also do following improvement.
Further, the mark module of the local valid pixel collection of described raster data comprises:
Whether local valid pixel judges judge module, be the value of invalid grid pixel for the value that judges successively the each pixel of described raster data; If so, call invalid grid pixel judge mark module, if not, local effectively grid pixel judge mark module called;
Invalid grid pixel mark module, for the value of determining that the value of described raster data pixel is invalid grid pixel, and by described grid 0 mark;
Local effectively grid pixel mark module, is effective grid pixel for the value of determining described raster data pixel, and by described grid 1 mark;
Mark raster data generation module, for recording in certain sequence the mark of all pixels of described raster data, forms mark raster data;
Mark raster data compression module, redistributes for described mark raster data is accounted for to 1bit storage space according to every pixel; Starting continuous 32 according to the initial number from described mark raster data is one group, and by 32 every group the flag data of 4 bytes get the mode that its decimal integer value stores and compress, obtain compact token raster data.
Further, described read module comprises:
Memory allocation module, be used for according to the value of the pixel width of the essential information of described raster data, pixel height, invalid grid pixel and pixel data type, distribute and the two-dimensional matrix storage space of the corresponding size of described raster data, each grid of two-dimensional matrix is " untreated state " described in initialization;
Decompress(ion) compact token raster data module, for compact token raster data described in decompress(ion), gives described two-dimensional matrix by mark raster data, marks invalid grid pixel and the local effectively position of grid pixel in described two-dimensional matrix;
Invalid grid pixel packing module, for the relevant position of the invalid grid pixel of described two-dimensional matrix being inserted to the value of invalid grid pixel, and grid state is set to " processed state ";
Local effectively grid pixel packing module, for the pixel value serial data of described effective grid pixel being inserted in certain sequence to the grid of two-dimensional matrix " untreated state ", obtains the raster data restoring at receiving end.
The invention also discloses a kind of local effectively transmission method of raster data that comprises,
Server adopts a kind of access method that comprises local effective raster data of the present invention to process raster data, using the essential information of the pixel value serial data of the compact token raster data obtaining, effective grid pixel, raster data as connection object, transmit, send client to, client is stored accordingly and reads compact token raster data, the pixel value serial data of effective grid pixel, the essential information of raster data.
The present invention is labeled as cost by the effective grid pixel of additional records collection, exchanges for take local effectively grid pixel collection in raster data as storage or connection object, thereby saves a large amount of storage spaces or transmission volume; The present invention does not relate to the computing of too much complexity, therefore in actual items application, can guarantee the efficiency of real time data processing; This method is one of them aspect of the structure organization based on raster data, do not affect the last handling process to data, for example, by the method for the invention, obtain after the actual pixels Value Data string of effective grid pixel collection label information and local effectively grid pixel collection, all can carry out again the last handling processes such as data compression, coding, further reduce data storage and transmission quantity.
Accompanying drawing explanation
Fig. 1 is a kind of local effectively process flow diagram of the access method embodiment of raster data that comprises;
Fig. 2 is pixel value schematic diagram in the raster data rectangular area of Chinese soil classification embodiment;
Fig. 3 is the pixel region schematic diagram that the raster data rectangular area of Chinese soil classification embodiment needs actual transmissions;
Fig. 4 is part effective grid pixel collection mark raster data schematic diagram of Chinese soil classification embodiment;
Fig. 5 is part effective grid pixel collection mark raster data schematic diagram of Chinese soil classification embodiment;
Fig. 6 is the actual pixel value serial data of the effective grid pixel collection in part of Chinese soil classification embodiment;
Fig. 7 is a kind of local effectively structured flowchart of the access device embodiment of raster data that comprises;
Fig. 8 is a kind of local effectively read module example structure block diagram of the access device of raster data that comprises.
Embodiment
Below in conjunction with accompanying drawing, principle of the present invention and feature are described, example, only for explaining the present invention, is not intended to limit scope of the present invention.
Inventor is according to the experience in actual items, find the raster data transmitting in application system, often comprise part engineering noise pixel data, " invalid " pixel data here, the definite pixel value that refers to not know in actual applications these pixels, these inactive pixels are mainly derived from following several situation:
A, in data production run, remote sensing satellite may be subject to the impact such as cloud layer and atmospheric dust, causes part grid pixel not get numerical value, general data manufacturer is labeled as without data value.This part without data value pixel distribute there is randomness, quantity also has randomness.
B, in actual application, data manufacturer need be labeled as partial-pixel value without data value.For example, on the make when state's vegetation coverage raster data, to waters such as the rivers and lakes in Chinese scope, owing to there is no the implication of vegetation, these pixel values need to be labeled as without data value.This part without data value pixel with applicable cases difference, its Numerical Distribution differs greatly, and has certain randomness.
Grid pixel outside c, specific region is labeled as without data value by data manufacturer.As while issuing Chinese soil classification raster data, to the pixel outside Chinese national boundaries in grid rectangular area, be generally all labeled as without data value.This class generally distributes concentrated without data value and quantity is more.
In actual applications, the invalid grid pixel collection of raster data can be by situation a, b, any one or more common generation of c.In these raster datas, comprising distributes has invalid grid pixel collection randomness, that quantity has randomness and local effectively grid pixel collection (for operating grid, all pixels outside invalid grid pixel collection are local effectively grid pixel).
Know-why of the present invention is to the effective grid pixel collection of the part in raster data, and point three parts are carried out information storage: the one, and the label information of local effective pixel collection, is effective pixel to distinguish which pixel in rectangular area, which is invalid pixel; The 2nd, the actual pixels value information of the local effective pixel collection of arranging by particular order; The 3rd, the essential information of raster data.This three partial information is transmitted respectively, when reception, carried out dissection process by corresponding method, again obtain rectangular area raster data.
The local valid pixel collection label information disposal route adopting in the present invention, no matter how can realize the distribution of invalid grid pixel, quantity how, and the duplicate target of compression efficiency can meet raster data in actual production and often have the situation without data value of a large amount of dispersivenesses.
Fig. 1 is a kind of local effectively process flow diagram of the access method embodiment of raster data that comprises, and with reference to diagram 1, a kind of effectively access method embodiment of raster data of part that comprises of the present invention, specifically comprises:
Understand the application for convenience of technician, now set forth take Chinese soil classification raster data processing as embodiment.Fig. 2 is pixel value schematic diagram in the raster data rectangular area of Chinese soil classification embodiment, as shown in Figure 2, represent in the raster data of Chinese soil classification, known raster data is R, the rectangular area A of this raster data is to be W pixel by width, is highly that H pixel forms, total pixel N=W*H, the data storage of each grid pixel needs m byte, and needing altogether storage space is Q=N*m byte.In the N of this raster data pixel, include invalid grid pixel.Wherein, judge whether certain pixel is that invalid grid pixel is to distinguish with the value of this pixel.In actual raster data, carry out a certain numerical value representative of corresponding appointment " invalid grid pixel " according to the pixel data type of grid.For example, represent that in the raster data of Chinese soil classification, China has 46 kinds of land types, so store the land classification number of each pixel by the byte data type of 1 byte, classification number, since 1, gets 46; Represent that with numerical value 0 this pixel is invalid grid pixel; The rectangular area pixel value of this raster data as shown in Figure 2.As can be seen from Figure 2, the grid pixel that pixel value is 0 is invalid grid pixel, or they are for applying the region of loseing interest in, or are " without data value " pixel.Wherein, the pixel area that needs in other words actual transmissions that in the raster data rectangular area of this Chinese soil classification, application is concerned about is as schematic diagram 3, and Fig. 3 is the pixel region schematic diagram that the raster data rectangular area of Chinese soil classification embodiment needs actual transmissions;
As shown in Figure 3, in Fig. 3, the pixel value of the subregion in rectangular area beyond national boundaries is 0 at Fig. 2, belongs to invalid grid pixel; And the value that also has fraction grid pixel in Fig. 3 state boundary is 0 pixel value in Fig. 2, this partial-pixel is to have " without data value " pixel, certainly also belongs to invalid grid pixel.Like this, in Fig. 2, according to grid pixel value, we just can determine which is effective grid pixel, and which is invalid grid pixel.
It should be noted that different raster datas, according to the data type of pixel value, determine the numerical value of invalid grid pixel, value there is no mandatory provision (this is generally all determined by the department that manufactures of raster data, has corresponding explanation in the time selling data).Under general applicable cases, be 0 the integer of being greater than of 1 byte for pixel data type, desirable 0 or 255 are used as invalid grid pixel value; Be 0 the integer of being greater than of 4 bytes for pixel data type, desirable 0 or 9999999.9999(mono-plurality, all the other effective grid pixel values all can not have this value, can distinguish and not be normal grid pixel) be used as invalid grid pixel value;
Comprehensively above-mentioned, known conditions of the present invention is exactly the raster data in the Fig. 2 based on given, includes invalid grid pixel in its rectangular area, and the value of invalid grid pixel is determined.For convenience of narration, in Chinese soil classification embodiment, all suppose that the value of invalid grid pixel is 0.It is not 0 grid pixel collection that local effectively grid pixel collection is all pixel values in Fig. 2, and we change at present all modes of transmission of Fig. 2 grid pixel data, only this local effectively grid pixel collection are stored or are transmitted.
Described in step 1, the effective grid pixel in raster data and invalid grid pixel are carried out to mark, generate mark raster data; Mark raster data is compressed, obtains compact token raster data, in the realization of Chinese soil classification embodiment, specifically comprise:
In the raster data rectangular area in given Fig. 2, to the effective grid pixel in part, value is all set to 1; To invalid grid pixel, value is all set to 0.Like this, obtained a new raster data R2.Now all with known raster data R. raster data R2 as shown in Figure 4, Fig. 4 is part effective grid pixel collection mark raster data schematic diagram of Chinese soil classification embodiment for the wide W of R2 raster data pixel, high H.(note: raster data R2 is now intermediate operations result, does not need writing in files to store! )
To raster data R2, account for 1bit storage space by every pixel and redistribute pixel size and pixel value calculating.To the raster data R2 in Fig. 4, it has H capable, W row pixel.Suppose that the raster data after distributing is R3, its number of rows of picture elements is still that H is capable, but columns is the positive integer that is not less than W/32, and the pixel data type that is designated as C. raster data and is R3 is the integer of 4 bytes.
To any row r (1<=r<=H of raster data R3, line number is since 1), the value of its first pixel is x, the binary form of x is shown with 32bit position, from left to right, the 1st is the 1st train value (0 or 1) that in Fig. 3, r is capable, the 2nd is the 2nd train value (0 or 1) that in Fig. 3, r is capable, ..., by that analogy, the 32nd is the 32nd train value (0 or 1) that in Fig. 3, r is capable, like this, the binary representation of x is just decided, such as being 11100000111100111100000111100111, x value is metric 3774071271, to second pixel y, the binary form of y is shown with 32bit position, from left to right, the 1st is the 33rd train value (0 or 1) that in Fig. 3, r is capable, and the 2nd is the 34th train value (0 or 1) that in Fig. 3, r is capable ..., by that analogy, the 32nd is the 64th train value (0 or 1) that in Fig. 3, r is capable, and like this, the numerical value of y just can be decided (with decimal representation), in like manner, to the individual pixel value z of j (1<=j<=C), the binary form of z is shown with 32bit position, from left to right, takes from respectively (j-1) * 32+1 train value that in Fig. 3, r is capable ..., j*32 train value.If the pixel width W<C*32 of Fig. 3, needs that the right side binary digit of last row pixel value is got to 0 and supplies.
Process as stated above and obtain local effectively grid pixel collection mark raster data R3, as shown in Figure 4:
Here, raster data R3 width is C pixel, is highly H pixel, and every pixel data storage class is 4 byte integer, needs altogether C*H*4 bytes of storage space.By this raster data R3 be written in file or in other media, store or internal memory in pending Internet Transmission.
Through the processing of step 1, effective grid pixel in raster data R and invalid grid pixel can be determined, so, at this time only need separately effective raster data stored or transmitted just passable, abandon the processing of invalid grid pixel parts data, in the realization of Chinese soil classification embodiment, specific as follows:
To pixel in the rectangular area of raster data R in Fig. 2, from top to bottom, from left to right, pixel judges one by one, if its pixel value is that (illustrate above, the invalid grid pixel value of each raster data is all known numeric value to invalid grid pixel value, can be 0, also can be other specific numerical value), ignore this pixel value; Effective pixel value if, carries out record in order by this pixel value.Namely, in order all effective grid pixels are arranged, obtained a string numerical value.To the raster data in Fig. 2, after its local effective grid pixel collection processed in sequence, obtain series of values as shown in Figure 6, Fig. 6 is the actual pixel value serial data of the effective grid pixel collection in part of Chinese soil classification embodiment; The data type of each numerical value of Fig. 6 signal keeps identical with known raster data R, and each numerical value needs m bytes of storage space.The serial data of this particular order is write to stored medium file or internal memory, for storage or Internet Transmission.
In the realization of Chinese soil classification embodiment, specifically comprise:
In the time that needs are stored known raster data R, by the effective grid pixel collection mark raster data R3 writing in files R_Tag.file obtaining in above-mentioned steps one; Meanwhile, by the actual pixels Value Data string writing in files R_data.file of the effective grid pixel collection in step 2, the essential information of the raster data in step 3 is stored in to file R_info.file.
In the realization of Chinese soil classification embodiment, specifically comprise:
Application system first reads the essential information file R_info.file of raster data, obtain the rectangular area pixel width W of raster data, height H and pixel data type and invalid grid pixel value v, thereby in internal memory, clearly distribute corresponding two-dimensional matrix D ［ W ］ ［ H ］ (corresponding two-dimensional grid), each grid of this matrix of initialization is " untreated " state; File reading R_Tag.file again, by the inverse process in step 2, will resolve mark raster data R3, and to the grid value of tax v of invalid grid pixel corresponding in two-dimensional matrix D, and this grid state is set to " processed " state; File reading R_data.file again, sense data string a1 in order, a2, ..., an, to two-dimensional matrix D from left to right, from top to bottom, compose respectively the grid take the 1st state as " untreated ", the 2nd state is the grid of " untreated " ..., n state is the grid of " untreated ". so far, the capable W row of H grid in this two-dimensional matrix D all obtains assignment, i.e. corresponding rectangular grid data R.
In the time that needs carry out Internet Transmission to known raster data R, by the effective grid pixel collection mark raster data R3 write memory obtaining in above-mentioned steps one; Simultaneously, by the actual pixels Value Data string write memory of the effective grid pixel collection obtaining in step 2, carry out respectively network data transmission, in addition, essential information that need to this raster data, comprise pixel width, pixel height, invalid grid pixel value and pixel data type, the essential information of the raster data obtaining in step 3 is carried out to Internet Transmission simultaneously.At network reciever, with above-mentioned file storage, read same method and carry out dissection process, just can obtain in similar Fig. 1 raster data.
The performance evaluation that raster data processing method of the present invention is applied in Chinese soil classification embodiment is as follows:
The present invention is labeled as cost by the effective grid pixel of additional records collection, exchanges for take local effectively grid pixel collection in raster data rectangular area as storage or connection object, thereby saves a large amount of storage spaces or transmission volume.In the raster data rectangular area shown in Fig. 2, in actual items application, be only concerned about that state boundary that Fig. 3 illustrates, with effective grid pixel of interior (containing), that is to say, the invalid raster data outside state boundary almost accounts for the half of rectangular area pixel.Suppose that the raster data amount in Fig. 2 is Q byte.In the time adopting this method, the local effectively actual pixels Value Data string that grid pixel integrates, only as the required half data volume of commonsense method, is Q/2 byte; And the local effectively grid pixel collection mark raster data amount S of the record increasing, in the time that the every pixel of the grid in Fig. 2 is 1 byte, S=Q/8; In the time that the every pixel of the grid in Fig. 2 is 2 byte, S=Q/16; In the time that the every pixel of the grid in Fig. 2 is 4 byte, S=Q/32; Owing to increasing, the raster data essential information amount of storing is very little, takies the storage space less than 1Kb, negligible; In least favorable situation (S=Q/8), altogether need 5Q/8 byte, save 37.5% data volume; Under vantage (S=Q/32), altogether need 17Q/32 byte, save 46.8% data volume; Visible, this method can be saved a large amount of storages and transmission quantity, and invalid grid pixel is more, and efficiency is more obvious.
Raster data processing method of the present invention does not relate to the computing of too much complexity, therefore in actual items application, can guarantee the efficiency of real time data processing.
Described in this method, raster data processing method is one of them aspect of the structure organization based on raster data, does not affect the last handling process to data.For example, pressing the method for the invention, in Chinese soil classification embodiment, obtain, after the actual pixels Value Data string of effective grid pixel collection label information and local effectively grid pixel collection, all can carrying out again the last handling processes such as data compression, coding, further reduce data storage and transmission quantity.
The label information disposal route adopting in the present invention, though be pursue invalid grid pixel distribution how, quantity how, the duplicate target of compression efficiency, therefore, adopt the mark of every pixel is adopted to 1 to store, continuous 32 pixels need 32 an integer of 4 bytes carry out record.Like this, just can meet random distribution and the quantity problem of invalid grid pixel.Thereby, can meet raster data in actual production and often have the situation without data value of a large amount of dispersivenesses.
Better understand the raster data processing method described in the application for convenience of technician, another embodiment is provided below.
DEM (Digital Elevation Model: digital elevation model) raster data is the most basic geo-spatial data, in actual items, need the DEM raster data within the scope of storage and transmission regional, the resolution requirement of this raster data is 1 kilometer. the DEM raster data by mapping part producing issue is the rectangular area that wide 4841 pixels, high 4032 pixels form, and the data type of each grid pixel is the floating type numerical value of 4 bytes.In the time that this DEM raster data is issued, in this rectangular area and the extraneous grid pixel of state boundary, all assignment is-999999.9999, its implication is that this partial-pixel belongs to invalid grid pixel (because global actual earth's surface elevation is there is no-999999.9999 meters of these situations, so can be worth to represent that this pixel is to belong to invalid pixel with this.But unavailable 0 as invalid grid pixel value, because actual earth's surface elevation has 0 meter, cannot distinguish this pixel is 0 meter of elevation or invalid pixel).
In project, if store with applying the Tif raster data file layout of Arcgis software more widely in GIS industry, the storage space needing is Q=4841*4032*4 byte, about 74M data volume.Supplementary notes: Tif raster data file layout is not distinguish the Typical Representative of effective grid pixel and invalid grid pixel, to all pixel values, adopts unified mode to store, this is also the method that current all kinds of raster data storage adopts.
By the method for the invention, obtain local effectively grid pixel collection mark raster data DEM_R2 in step 2, this raster data is wide is 4841 pixels, height is 4032 pixels, wherein, grid pixel value is totally 9173889 of effective grid pixels of 1, accounting for 47%. of total rectangular area pixel then calculates according to redistribute pixel size and pixel value in step 2, obtain local effectively grid pixel collection mark raster data DEM_R3 (similar with Fig. 4), the wide of this raster data is 152 pixels, height is 4032 pixels, the integer storage of 4 bytes for every pixel, like this, additionally need to store effective grid pixel collection label information and need altogether E=152*4032*4 byte, be about 2.4M data volume, according to step 3, the actual pixels value string (serial data and Fig. 5 are similar) that obtains part effective grid pixel collection of this DEM raster data, has 9173889 numerical value, and each numerical value is the float type of 4 bytes, need altogether F=9173889*4 byte, be about 35M data volume, in addition, also need to store the essential information of DEM raster data, comprise that pixel width value, pixel height value, invalid grid pixel value and every pixel data type of this DEM raster data is float type.Because taking storage space, these essential informations are less than 1Kb, the calculation of can ignoring; , this method is stored this DEM raster data needs E+F=37.4M altogether, compares the nearly half data volume of Tif raster data file layout saving.
When this DEM raster data is carried out to Internet Transmission, identical with storage means principle and process above, different is write memory, rather than writing in files.
In actual items, applying this method result proves, memory space and the transmission volume of raster data are reduced greatly, and the performance of system is made to significant contribution.
Fig. 7 is a kind of local effectively structured flowchart of the access device embodiment of raster data that comprises, and as shown in Figure 7, the invention also discloses a kind of local effectively access device embodiment of raster data that comprises, and comprising:
Raster data module, for storing original raster data, comprises in described raster data in raster data and comprises effective grid pixel and invalid grid pixel, and the value of invalid grid pixel is data manufacturer's designated value;
The mark module of the local valid pixel collection of raster data, for effective grid pixel of described raster data and invalid grid pixel are carried out to mark, generates mark raster data; Described mark raster data is compressed, obtain compact token raster data;
The mark module of the local valid pixel collection of raster data comprises:
Whether local valid pixel judges judge module, be the value of invalid grid pixel for the value that judges successively the each pixel of described raster data; If so, call invalid grid pixel judge mark module, if not, local effectively grid pixel judge mark module called;
Invalid grid pixel mark module, for the value of determining that the value of described raster data pixel is invalid grid pixel, and by described grid 0 mark;
Local effectively grid pixel mark module, is effective grid pixel for the value of determining described raster data pixel, and by described grid 1 mark;
Mark raster data generation module, for recording in certain sequence the mark of all pixels of described raster data, forms mark raster data;
Mark raster data compression module, redistributes for described mark raster data is accounted for to 1bit storage space according to every pixel; Starting continuous 32 according to the initial number from described mark raster data is one group, and by 32 every group the flag data of 4 bytes get the mode that its decimal integer value stores and compress, obtain compact token raster data.
The actual pixel value module of local valid pixel collection, for the pixel value of effective grid pixel of described raster data is stored separately in certain sequence, abandons the processing of invalid grid pixel partial data, obtains the pixel value serial data of effective grid pixel;
The basic information module of raster data, for recording the essential information of described raster data, essential information comprises value and the pixel data type of pixel width, pixel height, invalid grid pixel;
Memory module, in the time that application system need to be stored raster data, using the essential information of the pixel value serial data of described compact token raster data, effective grid pixel, raster data as storage object, writing in files is stored;
Read module, for in the time that application system need to read the raster data of storage, the essential information of the raster data in first file reading, and according to the essential information of raster data, distribute the two-dimensional matrix storage space with the corresponding size of described raster data, according to the pixel value serial data of compact token raster data and effective grid pixel, two-dimensional matrix is processed again, obtained the raster data restoring.
Fig. 8 is a kind of local effectively read module example structure block diagram of the access device of raster data that comprises, and as shown in Figure 8, read module comprises:
Memory allocation module, be used for according to the value of the pixel width of the essential information of described raster data, pixel height, invalid grid pixel and pixel data type, distribute and the two-dimensional matrix storage space of the corresponding size of described raster data, each grid of two-dimensional matrix is " untreated state " described in initialization;
Decompress(ion) compact token raster data module, for compact token raster data described in decompress(ion), gives described two-dimensional matrix by mark raster data, marks invalid grid pixel and the local effectively position of grid pixel in described two-dimensional matrix;
Invalid grid pixel packing module, for the relevant position of the invalid grid pixel of described two-dimensional matrix being inserted to the value of invalid grid pixel, and grid state is set to " processed state ";
Local effectively grid pixel packing module, for the pixel value serial data of described effective grid pixel being inserted in certain sequence to the grid of two-dimensional matrix " untreated state ", obtains the raster data restoring at receiving end.
The above implementation step and method have only been expressed one embodiment of the present invention, and description is comparatively concrete and detailed, but can not therefore be interpreted as the restriction to the scope of the claims of the present invention.Do not departing under the prerequisite of patent design of the present invention, the distortion of doing and improvement should all belong to the protection domain of patent of the present invention.
Claims (7)
1. comprise a local effectively access method for raster data, it is characterized in that:
In raster data, comprise effective grid pixel and invalid grid pixel, the value of invalid grid pixel is data manufacturer's designated value;
Effective grid pixel in raster data and invalid grid pixel are carried out to mark, generate mark raster data; Mark raster data is compressed, obtain compact token raster data;
The pixel value of the effective grid pixel in raster data is stored separately in certain sequence, obtained the pixel value serial data of effective grid pixel;
Record the essential information of raster data, essential information comprises value and the pixel data type of pixel width, pixel height, invalid grid pixel;
In the time that application system need to be stored raster data, using the essential information of the pixel value serial data of described compact token raster data, effective grid pixel, raster data as storage object, writing in files is stored;
In the time that application system need read the raster data of storage, the essential information of the raster data in first file reading, and according to the essential information of raster data, distribute the two-dimensional matrix storage space with the corresponding size of described raster data, according to the pixel value serial data of compact token raster data and effective grid pixel, two-dimensional matrix is processed again, obtained the raster data restoring.
2. a kind ofly according to claim 1 comprise the local effectively access method of raster data, it is characterized in that, described effective grid pixel in raster data and invalid grid pixel are carried out to mark, generate mark raster data; Mark raster data is compressed, and the step that obtains compact token raster data comprises:
Whether the value that judges successively each pixel in described raster data is the value of invalid grid pixel;
If so, this grid pixel is invalid grid pixel, uses 0 mark;
If not, this grid pixel is effective grid pixel, uses 1 mark;
Record in certain sequence the mark of all pixels in described raster data, form mark raster data;
Described mark raster data is accounted for to 1bit storage space according to every pixel to be redistributed;
Be one group by the mark raster data after described redistributing according to starting continuous 32 from the initial number of mark raster data, and by 32 every group the flag data of 4 bytes get the mode that its decimal integer value stores and compress, obtain compact token raster data.
3. a kind ofly according to claim 1 comprise the local effectively access method of raster data, it is characterized in that,
Described according to the essential information of raster data, distribute the two-dimensional matrix storage space with the corresponding size of described raster data, according to the pixel value serial data of compact token raster data and effective grid pixel, two-dimensional matrix is processed, the step that obtains the raster data restoring comprises again:
According to the essential information of described raster data, distribute the two-dimensional matrix storage space with the corresponding size of described raster data; Each grid of this two-dimensional matrix of initialization is " untreated state ";
The described compact token raster data that decompresses, gives described two-dimensional matrix by described mark raster data, marks invalid grid pixel and the local effectively position of grid pixel in described two-dimensional matrix;
The relevant position of the invalid grid pixel of described two-dimensional matrix is inserted to the value of invalid grid pixel, and grid state is set to " processed state ";
Again the pixel value serial data of described effective grid pixel is inserted in certain sequence to the grid of " untreated state " in described two-dimensional matrix, obtained the raster data restoring at receiving end.
4. comprise a local effectively access device for raster data, it is characterized in that, comprising:
Raster data module, for storing original raster data, includes effective grid pixel and invalid grid pixel in described raster data, the value of invalid grid pixel is data manufacturer's designated value;
The mark module of the local valid pixel collection of raster data, for effective grid pixel of described raster data and invalid grid pixel are carried out to mark, generates mark raster data; Described mark raster data is compressed, obtain compact token raster data;
The actual pixel value module of local valid pixel collection, for the pixel value of effective grid pixel of described raster data is stored separately in certain sequence, abandons the processing of invalid grid pixel partial data, obtains the pixel value serial data of effective grid pixel;
The basic information module of raster data, for recording the essential information of described raster data, essential information comprises value and the pixel data type of pixel width, pixel height, invalid grid pixel;
Memory module, in the time that application system need to be stored raster data, using the essential information of the pixel value serial data of described compact token raster data, effective grid pixel, raster data as storage object, writing in files is stored;
Read module, for in the time that application system need to read the raster data of storage, the essential information of the raster data in first file reading, and according to the essential information of raster data, distribute the two-dimensional matrix storage space with the corresponding size of described raster data, according to the pixel value serial data of compact token raster data and effective grid pixel, two-dimensional matrix is processed again, obtained the raster data restoring.
5. a kind ofly according to claim 4 comprise the local effectively access device of raster data, it is characterized in that,
The mark module of the local valid pixel collection of described raster data comprises:
Whether local valid pixel judges judge module, be the value of invalid grid pixel for the value that judges successively the each pixel of described raster data; If so, call invalid grid pixel judge mark module, if not, local effectively grid pixel judge mark module called;
Invalid grid pixel mark module, for the value of determining that the value of described raster data pixel is invalid grid pixel, and by described grid 0 mark;
Local effectively grid pixel mark module, is effective grid pixel for the value of determining described raster data pixel, and by described grid 1 mark;
Mark raster data generation module, for recording in certain sequence the mark of all pixels of described raster data, forms mark raster data;
Mark raster data compression module, redistributes for described mark raster data is accounted for to 1bit storage space according to every pixel; Starting continuous 32 according to the initial number from described mark raster data is one group, and by 32 every group the flag data of 4 bytes get the mode that its decimal integer value stores and compress, obtain compact token raster data.
6. a kind ofly according to claim 4 comprise the local effectively access device of raster data, it is characterized in that,
Described read module comprises:
Memory allocation module, be used for according to the value of the pixel width of the essential information of described raster data, pixel height, invalid grid pixel and pixel data type, distribute and the two-dimensional matrix storage space of the corresponding size of described raster data, each grid of two-dimensional matrix is " untreated state " described in initialization;
Decompress(ion) compact token raster data module, for compact token raster data described in decompress(ion), gives described two-dimensional matrix by mark raster data, marks invalid grid pixel and the local effectively position of grid pixel in described two-dimensional matrix;
Invalid grid pixel packing module, for the relevant position of the invalid grid pixel of described two-dimensional matrix being inserted to the value of invalid grid pixel, and grid state is set to " processed state ";
Local effectively grid pixel packing module, for the pixel value serial data of described effective grid pixel being inserted in certain sequence to the grid of two-dimensional matrix " untreated state ", obtains the raster data restoring at receiving end.
7. one kind comprises the local effectively transmission method of raster data, it is characterized in that, server adopts any access method that comprises local effective raster data in claims 1 to 3 to process raster data, using the essential information of the pixel value serial data of the compact token raster data obtaining, effective grid pixel, raster data as connection object, transmit, send client to, client is stored accordingly and reads compact token raster data, the pixel value serial data of effective grid pixel, the essential information of raster data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410088639.XA CN103886045B (en) | 2014-03-12 | 2014-03-12 | Access method and device including local valid grid data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410088639.XA CN103886045B (en) | 2014-03-12 | 2014-03-12 | Access method and device including local valid grid data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103886045A true CN103886045A (en) | 2014-06-25 |
| CN103886045B CN103886045B (en) | 2017-02-15 |
Family
ID=50954937
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410088639.XA Active CN103886045B (en) | 2014-03-12 | 2014-03-12 | Access method and device including local valid grid data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103886045B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107977427A (en) * | 2017-11-28 | 2018-05-01 | 鲁东大学 | A kind of storage of classifying type raster data and read method |
| CN110505218A (en) * | 2019-08-07 | 2019-11-26 | 中国电子科技集团公司第二十八研究所 | Raster data self-adapting compressing transmission method and computer storage medium based on JSON |
| US10576112B2 (en) | 2015-01-02 | 2020-03-03 | Melaleuca, Inc. | Bacterial compositions |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060146362A1 (en) * | 2005-01-06 | 2006-07-06 | Romney Robert E | Digital image processing with inherent compression |
| CN102708109A (en) * | 2012-01-12 | 2012-10-03 | 北京地拓科技发展有限公司 | Raster data reading method and device and raster data writing method and device |
| CN102937962A (en) * | 2012-09-17 | 2013-02-20 | 北京地拓科技发展有限公司 | Method and device for writing and reading raster data containing no data |
-
2014
- 2014-03-12 CN CN201410088639.XA patent/CN103886045B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060146362A1 (en) * | 2005-01-06 | 2006-07-06 | Romney Robert E | Digital image processing with inherent compression |
| CN102708109A (en) * | 2012-01-12 | 2012-10-03 | 北京地拓科技发展有限公司 | Raster data reading method and device and raster data writing method and device |
| CN102937962A (en) * | 2012-09-17 | 2013-02-20 | 北京地拓科技发展有限公司 | Method and device for writing and reading raster data containing no data |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10576112B2 (en) | 2015-01-02 | 2020-03-03 | Melaleuca, Inc. | Bacterial compositions |
| CN107977427A (en) * | 2017-11-28 | 2018-05-01 | 鲁东大学 | A kind of storage of classifying type raster data and read method |
| CN107977427B (en) * | 2017-11-28 | 2021-08-24 | 鲁东大学 | Storage and reading method of classified raster data |
| CN110505218A (en) * | 2019-08-07 | 2019-11-26 | 中国电子科技集团公司第二十八研究所 | Raster data self-adapting compressing transmission method and computer storage medium based on JSON |
| CN110505218B (en) * | 2019-08-07 | 2021-11-16 | 中国电子科技集团公司第二十八研究所 | Grid data self-adaptive compression transmission method based on JSON and computer storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103886045B (en) | 2017-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Coastal wetland loss and environmental change due to rapid urban expansion in Lianyungang, Jiangsu, China | |
| Abubahia et al. | Advancements in GIS map copyright protection schemes-a critical review | |
| Yan et al. | A normalization-based watermarking scheme for 2D vector map data | |
| US9877031B2 (en) | System and method for multi-resolution raster data processing | |
| CN103886045A (en) | Access method and device including local valid grid data | |
| CN111523240B (en) | View analysis method based on DEM data | |
| CN103325081B (en) | Insertion and extracting method based on spatial domain image digital watermark | |
| Qiu et al. | High-payload reversible watermarking scheme of vector maps | |
| CN1808068A (en) | Differential type geographic location information coding algorithm based on region benchmark | |
| CN103093413B (en) | A kind of remote sensing image data write and the method and apparatus read | |
| CN102103600B (en) | Map building method and map system | |
| CN102508853B (en) | Method and system for filling raster data | |
| Qiu et al. | Reversible watermarking algorithm of vector maps based on ECC | |
| CN102867023A (en) | Method for storing and reading grid data and device | |
| Elmustafa et al. | Understanding economic development in rural Africa using satellite imagery, building footprints and deep models | |
| Tong et al. | High-capacity and robust watermarking scheme for small-scale vector data | |
| CN102436504A (en) | System and method for processing road data based on dynamic segmentation technology | |
| CN108052629A (en) | A kind of quick extra large land determination methods based on high accuracy DEM data | |
| Wang et al. | A storage method for remote sensing images based on google s2 | |
| Niu et al. | GIS watermarking: hiding data in 2D vector maps | |
| Kuleli | The socio-economic significance of the Turkish coastal environment for sustainable development | |
| CN102915329B (en) | Method and device for writing and reading grid data | |
| CN107992555B (en) | Method for storing and reading raster data | |
| CN114048364A (en) | Disaster prediction layer generation method, device, equipment and storage medium | |
| CN102708109B (en) | Raster data reading method and device and raster data writing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210319 Address after: 518110 Longfa road 428 Huayuan street B24 Huayuan street b23b Longhua Street Longyuan community Longhua Street Longhua District Shenzhen City Guangdong Province Patentee after: Shenzhen Morning Intellectual Property Operations Co.,Ltd. Address before: 264001 Hongqi Middle Road, Zhifu District, Yantai, Shandong 186 Patentee before: LUDONG University Effective date of registration: 20210319 Address after: 201100 room 509, building 2, No. 508, Chundong Road, Minhang District, Shanghai Patentee after: SHANGHAI TAIYU INFORMATION TECHNOLOGY Co.,Ltd. Address before: 518110 Longfa road 428 Huayuan street B24 Huayuan street b23b Longhua Street Longyuan community Longhua Street Longhua District Shenzhen City Guangdong Province Patentee before: Shenzhen Morning Intellectual Property Operations Co.,Ltd. |
|
| TR01 | Transfer of patent right |