CN117093541B - Data extraction and archiving system for remote sensing data - Google Patents
Data extraction and archiving system for remote sensing data Download PDFInfo
- Publication number
- CN117093541B CN117093541B CN202311045891.8A CN202311045891A CN117093541B CN 117093541 B CN117093541 B CN 117093541B CN 202311045891 A CN202311045891 A CN 202311045891A CN 117093541 B CN117093541 B CN 117093541B
- Authority
- CN
- China
- Prior art keywords
- data
- target
- remote sensing
- frame
- refers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013075 data extraction Methods 0.000 title claims abstract description 11
- 238000004590 computer program Methods 0.000 claims abstract description 5
- 238000000034 method Methods 0.000 claims description 14
- 238000000605 extraction Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 4
- 238000013500 data storage Methods 0.000 claims 1
- 238000005516 engineering process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/113—Details of archiving
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0682—Tape device
Abstract
The invention provides a data extraction and archiving system of remote sensing data, which comprises: the target telemetry data, a processor and a memory storing a computer program which, when executed by the processor, performs the steps of: when receiving the filing instruction of the target remote sensing data, the target remote sensing data is filed in the target tape, and when receiving the extracting instruction of the target remote sensing data, the target data is extracted from the target tape, so that the working efficiency is improved.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a data extraction and archiving system of remote sensing data.
Background
The remote sensing technology is a technology for detecting and identifying a target by sensing electromagnetic waves, visible light and infrared rays reflected or radiated by the target from a long distance. In the field of remote sensing satellite image data extraction, long strip data generated through data input processing generally has an image coverage range of thousands to tens of thousands square kilometers, and a single file is tens of GB or even hundreds of GB. In the existing satellite image data extraction field, the technical core is to extract the whole satellite image data to a storage and then to perform subsequent data processing service.
In the prior art, when the data is required to be extracted, the whole strip data file needs to be recovered, and then the satellite standard product can be processed, so that the data extraction time is increased by the processing mode.
Disclosure of Invention
Aiming at the technical problems, the invention adopts the following technical scheme:
a data extraction and archiving system for remote sensing data, the system comprising: the target telemetry data, a processor and a memory storing a computer program which, when executed by the processor, performs the steps of:
s100, when an archiving instruction of target remote sensing data is received, archiving the target remote sensing data in a target tape;
s200, when receiving an extraction instruction of target remote sensing data, extracting target data from the target tape;
the step S200 further comprises the following steps:
s201, obtaining the data ID to be extracted and extraction information B= { B corresponding to the data ID to be extracted 1 ，B 2 ，B 3 }, wherein B is 1 Refers to the position of the initial frame corresponding to the data ID to be extracted, B 2 Refers to the position of the end frame corresponding to the data ID to be extracted, B 3 Refers to the frame length corresponding to the data ID to be extracted;
s202, when the storage state of the target remote sensing data is offline storage, acquiring the storage capacity D of each storage block in the target tape 0 ；
S203 according to D 0 Obtaining intermediate extraction information D= { D corresponding to the data ID to be extracted 1 ，D 2 ，D 3 }，D 1 Refers to the initial storage block position corresponding to the data ID to be extracted, D 2 Refers to the offset of the storage block corresponding to the data ID to be extracted, D 3 Refers to the length of the extracted data corresponding to the ID of the data to be extracted;
wherein D is 1 Meets the following conditions:
c is B 1 Storage in target tapeBlock position;
further, D2 meets the following conditions:
mod((((B 1 -1)*B 3 +a0+1)+c*D 0 +D 0 )，D 0 ) Mod () is a remainder function;
further, D 3 Meets the following conditions:
D 3 ＝(B 2 -B 1 +1)*B 3 ；
s204, extracting the coincidence D from the target tape 1 、D 2 And D 3 As target data.
The invention has at least the following beneficial effects:
magnetic tape is a storage medium that is sequentially stored and read in accordance with storage blocks, and thus the size of the storage blocks needs to be taken into account when extracting target data from the target magnetic tape. The storage block size of the target tape is D 0 Refers to the total number of bytes capable of being stored in one storage block as D 0 . When the target tape is required to be cut, firstly, calculating the position D of a storage block of a start frame of data corresponding to the data ID to be extracted in the target tape 1 Then calculating to obtain the position of the initial frame of the data corresponding to the data ID to be extracted in the initial storage block, and then calculating to obtain the data length D corresponding to the data ID to be extracted 3 . Finally according to D 1 、D 2 、D 3 The data on the target magnetic tape is extracted to obtain the target data, so that the accurate extraction of the target data is realized, the target data is not required to be extracted after the data on the target magnetic tape are completely recovered, the working steps are reduced, and the working efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart of a data extraction and archiving system for remote sensing data according to an embodiment of the present invention.
Fig. 2 is a flow chart of another embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
The embodiment provides a data extraction and archiving system of remote sensing data, the system comprises: target telemetry data, a processor and a memory storing a computer program which, when executed by the processor, performs the steps of, as shown in fig. 1:
and S100, when receiving an archiving instruction of the target remote sensing data, archiving the target remote sensing data in the target tape.
Specifically, the target remote sensing data refers to strip data stored in an online storage medium; it can be understood that: the strip data are data acquired by remote sensing satellites during remote sensing.
Specifically, the archiving instruction refers to an instruction for archiving target telemetry data from an online storage medium to a magnetic tape.
Specifically, the step S100 further includes the following steps:
s101, acquiring a target archive information list A= { A corresponding to the target remote sensing data 1 ，A 2 ，……，A i ，……A n }，Ai＝{A i 1 ，A i 2 ，A i 3 }，A i 1 Refers to the initial byte position A in the metadata corresponding to the ith target frame data in the target remote sensing data i 2 Refers to the element corresponding to the ith target frame data in the target remote sensing dataFrame length in data, A i 3 The method is that an archive identifier is in metadata corresponding to the ith target frame data in the target remote sensing data, and i= … … n, wherein n is the total frame number of the target frame data in the target remote sensing data.
Further, A i 1 Meets the following conditions:
A i 1 ＝a+(b i -1)*A i 2 wherein a refers to the frame header length, b i The position of the initial frame of the ith target frame data in the target remote sensing data in the strip data is referred to.
Specifically, the A i 2 Are all equal.
When the target remote sensing data is archived from the online storage medium to the magnetic tape, the target archiving information list A, A corresponding to the target remote sensing data is obtained simultaneously i The starting byte position of the metadata corresponding to the ith target frame of the target remote sensing data, the frame length of the metadata and the corresponding archiving identification. When the target remote sensing data in the magnetic tape is required to be recovered, the starting byte position and the frame length of the metadata corresponding to each target frame data can be obtained by reading the target filing information list, so that the metadata corresponding to the target frame data is convenient to recover.
Furthermore, the type of the archiving identifier is a hash value, and any method for acquiring the data hash value in the prior art is known by a person skilled in the art, and falls into the protection scope of the present invention, and is not described herein again; the method for obtaining the data hash value can adopt an MD5 method.
S102, A is i The first intermediate file is stored as follows: the first intermediate file refers to a file for storing metadata corresponding to the target frame data.
Further, the format of the first intermediate file is xml format.
S103, when A i When the corresponding target frame data is used as the data to be frame, an intermediate archive information list A0= { A corresponding to the data to be frame is obtained from A 1 ，A 2 ，……，A r ，……，A i-1 }，A r Refers to the r-th intermediate archive information, which refers to the archive information corresponding to the frame data to be transmitted.
S104, when A r 3 ＝A i 3 When A is r Corresponding pointer information is archived in the target tape.
S105, when A r 3 ≠A i 3 At the time, A i The corresponding first intermediate file is archived in the target tape.
The hash value can be used for converting the data with longer length into the character string with shorter length, fixed and not easy to repeat through the algorithm, and when A is used for i When the first intermediate file is stored in the first intermediate file, firstly judging A i Whether or not the archive identification of metadata corresponding to the previous target frame is identical to the archive identification of metadata corresponding to the previous target frame. If the archiving identification is the same as the archiving identification of any metadata, only the pointer information corresponding to the metadata is stored in the target tape, and all information is not required to be stored in the target tape, so that the occupation of the space of the target tape is reduced. When the metadata corresponding to the target frame data needs to be recovered, the target frame data can be found out from the over-read pointer information i Identical A r Will not be to A i Has an influence on the recovery of (a).
S200, when receiving an extraction instruction of target remote sensing data, extracting target data from the target tape.
Specifically, the extraction instruction refers to an instruction that will extract the telemetry data required by the user from the magnetic tape.
Specifically, the step S200 further includes the following steps:
s201, obtaining the data ID to be extracted and extraction information B= { B corresponding to the data ID to be extracted 1 ，B 2 ，B 3 }, wherein B is 1 Refers to the position of the initial frame corresponding to the data ID to be extracted, B 2 Refers to the position of the end frame corresponding to the data ID to be extracted, B 3 Refers to the frame length corresponding to the data ID to be extracted.
Further, the data ID to be extracted refers to a unique identifier of the data to be extracted, where the data to be extracted refers to remote sensing data to be extracted by a user, for example, the data ID to be extracted is a name of the data to be extracted.
S202, when the storage state of the target remote sensing data is offline storage, acquiring the storage capacity D of each storage block in the target tape 0 。
S203 according to D 0 Obtaining intermediate extraction information D= { D corresponding to the data ID to be extracted 1 ，D 2 ，D 3 }，D 1 Refers to the initial storage block position corresponding to the data ID to be extracted, D 2 Refers to the offset of the storage block corresponding to the data ID to be extracted, D 3 Refers to the extracted data length corresponding to the data ID to be extracted.
Further, D 1 Meets the following conditions:
c is B 1 At the storage block location in the target tape.
Further, D2 meets the following conditions:
mod((((B 1 -1)*B 3 +a0+1)+c*D 0 +D 0 )，D 0 ) Mod () is a remainder function.
Further, D 3 Meets the following conditions:
D 3 ＝(B 2 -B 1 +1)*B 3 。
s204, extracting the coincidence D from the target tape 1 、D 2 And D 3 As target data.
In the above, the magnetic tape is a storage medium in which storage blocks are sequentially stored and read, and thus the size of the storage blocks needs to be taken into consideration when extracting target data from the target magnetic tape. The storage block size of the target tape in this embodiment is D 0 Refers to the total number of bytes capable of being stored in one storage block as D 0 . When the target tape is required to be cut, firstly, calculating the position D of a storage block of a start frame of data corresponding to the data ID to be extracted in the target tape 1 After which calculationObtaining the position of the initial frame of the data corresponding to the data ID to be extracted in the initial storage block, and then calculating to obtain the data length D of the data corresponding to the data ID to be extracted 3 . Finally according to D 1 、D 2 、D 3 The data on the target magnetic tape is extracted to obtain the target data, so that the accurate extraction of the target data is realized, the target data is not required to be extracted after the data on the target magnetic tape are completely recovered, the working steps are reduced, and the working efficiency is improved.
S205, when the storage state of the target remote sensing data is online storage, according to B 1 And B 3 Determining a starting byte position B corresponding to the data ID to be extracted 0 Data length B corresponding to data ID to be extracted 4 。
Further, B 0 Meets the following conditions:
B 0 ＝a0+1+(B 1 -1)*B 3 wherein a0 refers to a frame header length corresponding to the data ID to be extracted.
Further, B 4 Meets the following conditions:
B 4 ＝(B 2 -B 1 +1)*B 3 。
s206, extracting the coincidence B0 and B from the target tape 4 As target data.
When the storage state of the target remote sensing data is online storage, the method comprises the following steps of 0 Cut B at byte position of (2) 4 The data of the length is taken as target data.
Specifically, any method for determining the storage state of remote sensing data in the prior art is known to those skilled in the art, which is not described herein in detail, and for example, the storage state is determined by using the remote sensing data extension attribute.
In a specific embodiment, the step S200 further includes the following steps:
and S300, storing the target data in a preset storage area, wherein the storage mode of the target data is one of a distributed storage mode, an object storage mode or a local storage mode.
S400, when the target data is stored in the preset storage area, the first metadata (B) 0 、B 1 、B 2 、B 3 And B 4 ) Or second metadata (D of the target data 0 ，D 1 ，D 2 ，D 3 ) Stored in the second intermediate file, it can be understood that: the second intermediate file refers to a file for storing metadata corresponding to the target data.
Specifically, the format of the second intermediate file is xml format.
S500, storing the second intermediate file in the preset storage area.
And when the target data needs to be recovered later, the second intermediate file is read to obtain the first metadata or the second metadata of the target data, so that the target data can be recovered conveniently.
In another specific embodiment, the computer program, when executed by a processor, further performs the following steps, as shown in fig. 2:
s1, acquiring a target remote sensing data list H= { H 1 ，H 2 ，……，H x ，……，H p }，H x The x-th target remote sensing data is referred to, and x= … … p, p is the total number of the target remote sensing data.
S2, according to H, obtaining a first intermediate file list H0= { H0 corresponding to H 1 ，H0 2 ，……，H0 x ，……，H0 p }，H0 x Refers to H x A corresponding first intermediate file.
Specifically, the format of the first intermediate file is xml format.
Specifically, the step S2 further includes the following steps of obtaining H0 x ：
S21, obtaining H x Corresponding target archive information list C x ＝{C x 1 ，C x 2 ，……，C x i ，……C x n }， Refers to H x The starting byte position in the metadata corresponding to the ith target frame data,/for the metadata>Refers to H x Metadata mid-frame length corresponding to the ith target frame data,/and the like>Refers to H x The metadata corresponding to the ith target frame data is filed with an identification.
Further, the method comprises the steps of,meets the following conditions:
where a is the frame header length and b is H x Corresponding start frame position.
When the target remote sensing data list H is obtained, the first intermediate file list H0 is obtained at the same time, according to H x Obtaining C x Obtaining H x The initial byte position, the frame length and the archiving identification of the metadata corresponding to the ith target frame data are convenient for recovering H through H0 when the target remote sensing data are recovered later.
Furthermore, the type of the archiving identifier is a hash value, and any method for acquiring the data hash value in the prior art is known by a person skilled in the art, and falls into the protection scope of the present invention, and is not described herein again; the method for obtaining the data hash value can adopt an MD5 method.
S22, will (C) 1 ，C 2 ……C x ) Write into key file and regard key file as H0 x It can be understood that: the key file is a file for storing metadata corresponding to the target remote sensing data.
Further, the format of the first intermediate file is xml format.
Further, C 1 To C x SQL statements are adopted in writing the key file.
S3, acquiring a key archive information list G= { G corresponding to H according to H and H0 1 ，G 2 ，……，G x ，……，G p },G x ＝{H x ，H0 x ，G0 x }，G0 x Is H x Identification of the corresponding target tape.
S4, G x Sequentially writing into the storage area of the target tape to generate U= { H 1 ，H0 1 ，G0 1 ，……，H x ，H0 x ，G0 x ，……，H p ，H0 p ，G0 p }。
S5, when a request for recovering target remote sensing data is received, H0 in U is recovered p Analyzing the corresponding SQL sentences to recover target remote sensing data; those skilled in the art will know that any method for parsing SQL statement belongs to the protection scope of the present invention, and will not be described in detail herein.
Above, H0 x Including all the first intermediate files in the first x files, when the target remote sensing data needs to be recovered later, only H0 in U is needed p And reading the metadata to obtain the initial byte position, the frame length and the archiving identification of the corresponding metadata of all the target remote sensing data in the H, and recovering the metadata according to the information, so that the workload of recovering the data is reduced, and the working efficiency is improved.
While certain specific embodiments of the invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the invention. Those skilled in the art will also appreciate that many modifications may be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.
Claims (10)
1. A data extraction and archiving system for remote sensing data, the system comprising: the target telemetry data, a processor and a memory storing a computer program which, when executed by the processor, performs the steps of:
s100, when an archiving instruction of target remote sensing data is received, archiving the target remote sensing data in a target tape;
s200, when receiving an extraction instruction of target data, extracting the target data from the target tape;
the step S200 further comprises the following steps:
s201, obtaining the data ID to be extracted and extraction information B= { B corresponding to the data ID to be extracted 1 ，B 2 ，B 3 }, wherein B is 1 Refers to the position of the initial frame corresponding to the data ID to be extracted, B 2 Refers to the position of the end frame corresponding to the data ID to be extracted, B 3 Refers to the frame length corresponding to the data ID to be extracted;
s202, when the storage state of the target remote sensing data is offline storage, acquiring the storage capacity D of each storage block in the target tape 0 ；
S203 according to D 0 Obtaining intermediate extraction information D= { D corresponding to the data ID to be extracted 1 ，D 2 ，D 3 }，D 1 Refers to the initial storage block position corresponding to the data ID to be extracted, D 2 Refers to the offset of the storage block corresponding to the data ID to be extracted, D 3 Refers to the length of the extracted data corresponding to the ID of the data to be extracted;
wherein D is 1 Meets the following conditions:
D 1 = ，/>is a downward rounding function; c is B 1 Storage block locations in the target tape;
further, D 2 Meets the following conditions:
mod（（（（B 1 -1）B 3 +a0+1）+c/>D 0 +D 0 ），D 0 ) A0 refers to a frame header length corresponding to the data ID to be extracted, and mod () is a remainder function;
further, D 3 Meets the following conditions:
D 3 =（B 2 -B 1 +1）B 3 ；
s204, extracting the coincidence D from the target tape 1 、D 2 And D 3 As target data.
2. The system of claim 1, wherein the system further comprises the steps of:
s205, when the storage state of the target remote sensing data is online storage, according to B 1 And B 3 Determining a starting byte position B corresponding to the data ID to be extracted 0 Data length B corresponding to data ID to be extracted 4 ；
Further, B 0 Meets the following conditions:
B 0 =a0+1+（B 1 -1）B 3 ；
further, B 4 Is in accordance with the followingConditions are as follows:
B 4 =（B 2 -B 1 +1）B 3 ；
s206, extracting the coincidence B0 and B from the target tape 4 As target data.
3. The system of claim 2, wherein the system further comprises the steps of:
and S300, storing the target data in a preset storage area, wherein the storage mode of the target data is one of a distributed storage mode, an object storage mode or a local storage mode.
4. A system according to claim 3, characterized in that the system further comprises the steps of:
s400, when the target data is stored in the preset storage area, the first metadata (B) 0 ，B 1 ，B 2 ，B 3 ，B 4 ) Or second metadata (D of the target data 0 ，D 1 ，D 2 ，D 3 ) The method comprises the steps of storing the metadata in a second intermediate file, wherein the second intermediate file is used for storing metadata corresponding to target data;
s500, storing the second intermediate file in the preset storage area.
5. The system of claim 1, wherein the system further comprises the steps of:
s1, acquiring a target remote sensing data list H= { H 1 ，H 2 ，……，H x ，……，H p }，H x The x-th target remote sensing data is referred to, x= … … p, p is the total number of the target remote sensing data;
s2, according to H, obtaining a first intermediate file list H0= { H0 corresponding to H 1 ，H0 2 ，……，H0 x ，……，H0 p }，H0 x Refers to H x A corresponding first intermediate file;
s3, acquiring a key archive information list G= { G corresponding to H according to H and H0 1 ，G 2 ，……，G x ，……，G p },G x ={H x ，H0 x ，G0 x }，G0 x Is H x Identification of the corresponding target tape;
s4, G x Sequentially writing into the storage area of the target tape to generate U= { H 1 ，H0 1 ，G0 1 ，……，H x ，H0 x ，G0 x ，……，H p ，H0 p ，G0 p }；
S5, when a request for recovering target remote sensing data is received, H0 in U is recovered p And analyzing the corresponding SQL sentences to restore the target remote sensing data.
6. The system of claim 5, wherein S2 further comprises the steps of:
s21, obtaining H x Corresponding target archive information list C x ={C x 1 ，C x 2 ，……，C x i ，……C x n }，C x i ={C x i 1 ，C x i 2 ，C x i 3 }，C x i 1 Refers to H x The initial byte position, C, in the metadata corresponding to the ith target frame data x i 2 Refers to H x The frame length C in the metadata corresponding to the ith target frame data x i 3 Refers to H x Filing an identification in metadata corresponding to the ith target frame data, wherein i= … … n, n is the total frame number of the target frame data in the target remote sensing data;
wherein C is x i 1 Meets the following conditions:
C x i 1 =a+1+（b x -1）C x i 2 where a is the frame header length, b x Refers to H x A corresponding start frame position;
s22, will (C) 1 ，C 2 ……C x ) Write into key file and regard key file as H0 x The key file is a file for storing metadata corresponding to the target remote sensing data.
7. The system of claim 5, wherein the format of the first intermediate file is xml format.
8. The system of claim 6, wherein the C 1 To C x SQL statements are adopted in writing the key file.
9. The system according to claim 1, wherein said step S100 comprises the steps of:
s101, acquiring a target archive information list A= { A corresponding to the target remote sensing data 1 ，A 2 ，……，A i ，……A n }，A i ={A i 1 ，A i 2 ，A i 3 }，A i 1 Refers to the initial byte position A in the metadata corresponding to the ith target frame data in the target remote sensing data i 2 Refers to the frame length A in the metadata corresponding to the ith target frame data in the target remote sensing data i 3 The method is characterized in that a file identifier is arranged in metadata corresponding to the ith target frame data in the target remote sensing data, wherein i= … … n, n is the total frame number of the target frame data in the target remote sensing data;
further, A i 1 Meets the following conditions:
A i 1 =a+（b i -1）A i 2 wherein a refers to the frame header length, b i The position of the initial frame of the ith target frame data in the target remote sensing data in the strip data is referred to;
s102, A is i Storing the metadata into a first intermediate file, wherein the first intermediate file is used for storing metadata corresponding to target frame data;
s103, when A i When the corresponding target frame data is taken as frame data to be archived, an intermediate archiving information list A0= { A corresponding to the frame data to be archived is obtained from A 1 ，A 2 ，……，A r ，……，A i-1 }，A r The data storage method comprises the steps of referring to the r-th intermediate archiving information, wherein the intermediate archiving information refers to archiving information corresponding to frame data to be archived;
s104, when A r 3 =A i 3 When A is r Corresponding pointer information is archived in the target tape;
s105, when A r 3 ≠A i 3 At the time, A i The corresponding first intermediate file is archived in the target tape.
10. The system according to claim 9, wherein: the type of archive identification is a hash value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311045891.8A CN117093541B (en) | 2023-08-18 | 2023-08-18 | Data extraction and archiving system for remote sensing data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311045891.8A CN117093541B (en) | 2023-08-18 | 2023-08-18 | Data extraction and archiving system for remote sensing data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117093541A CN117093541A (en) | 2023-11-21 |
| CN117093541B true CN117093541B (en) | 2024-02-23 |
Family
ID=88782393
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311045891.8A Active CN117093541B (en) | 2023-08-18 | 2023-08-18 | Data extraction and archiving system for remote sensing data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117093541B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1999040587A1 (en) * | 1998-02-05 | 1999-08-12 | Danmere Limited | Video media control system |
| CN102522088A (en) * | 2011-11-25 | 2012-06-27 | 展讯通信（上海）有限公司 | Decoding method and device of audio frequency |
| CN102945679A (en) * | 2012-09-29 | 2013-02-27 | 华为技术有限公司 | Video data processing method and video data read-write equipment |
| CN103812934A (en) * | 2014-01-28 | 2014-05-21 | 浙江大学 | Remote sensing data publishing method based on cloud storage system |
| WO2018076952A1 (en) * | 2016-10-24 | 2018-05-03 | 杭州海康威视数字技术股份有限公司 | Method and apparatus for storage and playback positioning of video file |
| CN108733344A (en) * | 2018-05-28 | 2018-11-02 | 深圳市道通智能航空技术有限公司 | Data read-write method, device and circle queue |
| CN115129257A (en) * | 2022-07-08 | 2022-09-30 | 深圳市正浩创新科技股份有限公司 | Data reading and writing method, electronic equipment and computer readable storage medium |
| CN115525602A (en) * | 2021-06-25 | 2022-12-27 | 华为技术有限公司 | Data processing method and related device |
| CN117076200A (en) * | 2023-08-18 | 2023-11-17 | 北京天华星航科技有限公司 | Remote sensing data recovery method, device and storage medium based on metadata |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8156136B2 (en) * | 2008-10-16 | 2012-04-10 | The Curators Of The University Of Missouri | Revising imagery search results based on user feedback |
-
2023
- 2023-08-18 CN CN202311045891.8A patent/CN117093541B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1999040587A1 (en) * | 1998-02-05 | 1999-08-12 | Danmere Limited | Video media control system |
| CN102522088A (en) * | 2011-11-25 | 2012-06-27 | 展讯通信（上海）有限公司 | Decoding method and device of audio frequency |
| CN102945679A (en) * | 2012-09-29 | 2013-02-27 | 华为技术有限公司 | Video data processing method and video data read-write equipment |
| CN103812934A (en) * | 2014-01-28 | 2014-05-21 | 浙江大学 | Remote sensing data publishing method based on cloud storage system |
| WO2018076952A1 (en) * | 2016-10-24 | 2018-05-03 | 杭州海康威视数字技术股份有限公司 | Method and apparatus for storage and playback positioning of video file |
| CN108733344A (en) * | 2018-05-28 | 2018-11-02 | 深圳市道通智能航空技术有限公司 | Data read-write method, device and circle queue |
| CN115525602A (en) * | 2021-06-25 | 2022-12-27 | 华为技术有限公司 | Data processing method and related device |
| CN115129257A (en) * | 2022-07-08 | 2022-09-30 | 深圳市正浩创新科技股份有限公司 | Data reading and writing method, electronic equipment and computer readable storage medium |
| CN117076200A (en) * | 2023-08-18 | 2023-11-17 | 北京天华星航科技有限公司 | Remote sensing data recovery method, device and storage medium based on metadata |
Non-Patent Citations (7)
| Title |
|---|
| 一种遥感图像多级感兴趣区域的压缩方法;杜维涛;刘方;陶永宽;蔡理金;;计算机仿真;20080915(第09期);221-225 * |
| 基于网格服务的遥感图像元数据自动获取;程妍妍;崔宾阁;;电脑知识与技术;20100125(03);63-66 * |
| 实现栅格图形和图像数据矢量化提取的游程轮廓追踪法;谢顺平, 都金康, 王结臣;遥感学报;20041010(第05期);82-87 * |
| 支持SAR系统高分辨率实时成像的实时系统解决方案;王青, 邹勇, 舒国强, 淮晓永, 李明树;计算机研究与发展;20030130(01);40-47 * |
| 支持遥感影像在轨处理的闪存缓存管理;罗平;中国优秀硕士学位论文全文数据库 信息科技辑;20181215(第12期);I140-972 * |
| 机载SAR实时成像系统自聚焦方法研究;韩冀中, 韩承德, 王贞松;计算机研究与发展;20030130(第01期);13-19 * |
| 遥感卫星数据磁带存储系统的技术研究与实现;吴广;杨仁忠;唐梦辉;;微计算机信息;20100325(09);38-39+42 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117093541A (en) | 2023-11-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105009067B (en) | Managing operations on units of stored data | |
| EP3620931B1 (en) | Searching for data using superset tree data structures | |
| CN101326492B (en) | Methods and systems for updating content including a compressed version | |
| KR101456757B1 (en) | Method and Apparatus for recovering deleted data for SQLite database | |
| JP6491605B2 (en) | Manage operations on stored data units | |
| JP2009539169A5 (en) | ||
| US20160034201A1 (en) | Managing de-duplication using estimated benefits | |
| CN102314697B (en) | Data type-based numeric data compression and decompression method | |
| CN108415986B (en) | Data processing method, device, system, medium and computing equipment | |
| CN117076200B (en) | Remote sensing data recovery method, device and storage medium based on metadata | |
| US20160147800A1 (en) | Data Processing Method and System and Client | |
| CN112380071B (en) | Method for quickly backing up NTFS file system | |
| EP3798842A1 (en) | Database deleted record recovery method and system | |
| CN112052120B (en) | Database deleted data recovery method and device | |
| CN117093541B (en) | Data extraction and archiving system for remote sensing data | |
| KR102275240B1 (en) | Managing operations on stored data units | |
| CN110874346A (en) | Compression scheme for floating point values | |
| CN113761059A (en) | Data processing method and device | |
| CN102890818A (en) | Method and device for reassembling joint photographic experts group (JPG) picture fragments based on thumbnail | |
| CN111309677A (en) | File management method and device of distributed file system | |
| CN112883000B (en) | Deformation monitoring radar data file storage method | |
| CN113076221B (en) | Data recovery method for MongoDB-MMAPv1 engine | |
| CN115049267A (en) | Wind turbine generator data processing method, system, equipment and computer medium | |
| US20220100385A1 (en) | Data reduction in block-based storage systems using content-based block alignment | |
| CN109271463B (en) | Method for recovering inodb compressed data of MySQL database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |