CN117171789A - File storage and access system and method based on file cutting - Google Patents
File storage and access system and method based on file cutting Download PDFInfo
- Publication number
- CN117171789A CN117171789A CN202311113362.7A CN202311113362A CN117171789A CN 117171789 A CN117171789 A CN 117171789A CN 202311113362 A CN202311113362 A CN 202311113362A CN 117171789 A CN117171789 A CN 117171789A
- Authority
- CN
- China
- Prior art keywords
- file
- block
- storage
- information
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 230000006835 compression Effects 0.000 claims abstract description 42
- 238000007906 compression Methods 0.000 claims abstract description 42
- 238000007796 conventional method Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention mainly relates to the technical field of electronic data processing. The method aims to solve the problems that the file with larger data volume after compression is inconvenient to store and the resource and time are wasted because the whole file is required to be acquired when only partial file data needs to be accessed for the file with larger data volume. The invention provides a file storage and access system and method based on file cutting, which stores file information into a file basic attribute table; cutting the file, storing the cut file blocks into a file storage pool, and storing file block information into a file block detail table; the user uploads the file compression code to the file storage system through the file access terminal, the file storage system checks the file compression code, and after the file compression code passes the check, the file storage system returns the corresponding file block to the file access terminal, so that the problem of inconvenient file storage with larger data volume is solved, and the dispersed storage space can be utilized as much as possible.
Description
Technical Field
The invention mainly relates to the technical field of electronic data processing, in particular to a file storage and access system and method based on file cutting.
Background
Electronic data is generally stored in a medium such as a magnetic disk in the form of a file, and there are two problems that a management party of the electronic data needs to solve. The first problem is how to store very large files; the second problem is how to achieve security sharing of data.
For the first problem, the conventional method is to compress the file to save the storage space and expand the capacity of the storage medium, and the disadvantage of this method is that the compression rate is limited, for example, the compression rate of the gzip method is between 5% and 10%, and the compression rate of the 7-Zip compression method with better compression rate can reach 90%, but for a large file, even if the compression rate is 90%, the file is still a very large file, and the storage in a storage medium still cannot be guaranteed.
For the second problem, the conventional method is to encrypt and transmit the file, and the disadvantage of this method is that advanced encryption is also at risk of being cracked. In addition, sometimes users only need to access part of file data instead of the whole file, and if the whole large file is transmitted in an encrypted manner, resources are wasted.
Disclosure of Invention
The invention aims to solve the technical problems
The file storage and access method based on file cutting is provided, and the problems of inconvenient file storage of the compressed file with larger data volume and resource and time waste caused by the fact that the whole file needs to be acquired when only partial file data needs to be accessed for the file with larger data volume are solved.
The invention solves the technical problems
A file storage and access system based on file cutting comprises a file storage system and a file access terminal;
the file storage system comprises a file block storage pool, a file information base, a user information base and a user key base;
the file block storage pool is used for storing file blocks after file cutting; the file information base comprises a file basic attribute table and a file block detail table, wherein the file basic attribute table is used for recording file information before cutting, and the file block detail table is used for recording file block information after cutting; the user information base is used for registering the user accessing the file; the user key library is used for generating a file access key according to the file information library and the information stored in the user information library;
the file access terminal is used for sending a file compression code to the file storage system, and after the file compression code is checked by the file storage system, the file storage system sends a corresponding file block to the file access terminal according to the compression code.
Further, the file information stored in the file basic attribute table comprises a file identifier, a file name, a file type identifier, a file size identifier and a file identification code;
the file identifier is used for searching the corresponding file, the file type identifier is used for marking the type of the file, the file size identifier is used for identifying the byte number of the file, and the file identification code is used for distinguishing the corresponding file from the rest files.
Further, the file block information stored in the file block detail table comprises a file block identifier, a file block size identifier, a file block sequence number and a file block storage address identifier;
the file block identifier is used for searching a corresponding file block; the file identifier is used for identifying the file to which the file block belongs, and the file block size identifier is used for identifying the byte number of the file block; the file block sequence number is used for marking the sequence number of the file block in the file block sequence, and the file block storage address identifier is used for identifying the storage address of the file block.
Further, the user information base includes a user identifier for recording the identity of the user.
Further, the file storage pool is a plurality of non-contiguous storage spaces in the distributed storage device.
According to the file storage and access system based on file cutting, the invention also provides a file storage and access method based on file cutting, which comprises the following steps:
storing the file information into a file basic attribute table of a file information base;
cutting the file, storing the cut file blocks into a file storage pool, and storing the cut file block information into a file block detail table of a file information base;
the user uploads the file compression code to the file storage system through the file access terminal, the file storage system checks the file compression code, and after the file compression code passes the check, the file storage system returns the corresponding file block to the file access terminal.
Further, the file compression code includes user information, file information, a set of accessed file blocks, and a file access key.
Further, the file storage system checking the file compression code comprises checking the user information, and checking whether the user is registered in the user information base; checking the file information, and checking whether the file exists in a file information base; and checking the file block information, checking whether the file block exists in a file block storage pool, and checking whether the file block information in a file block detail table is matched with the file block information in the compression code, and checking a file access key, wherein the file access key belongs to the user and the file.
Further, the method for cutting the file comprises the steps of equally cutting the file according to the size of the file or cutting the file according to the definition of the functional area of the data in the file.
The beneficial effects of the invention are that
When the file storage system and the method based on file cutting are used for storing files with larger data quantity, firstly, files are cut according to the characteristics of the files and the methods such as data function definition, file blocks are stored in a file storage pool, the files are cut into file blocks which are separated for storing the files with larger data quantity by using scattered storage resources, the problem that the files with larger data quantity are inconvenient to store in a smaller storage space is solved, and the scattered storage space can be utilized as much as possible.
In the process of accessing the file, only the file block data appointed in the file compression code can be transmitted, and compared with the process of transmitting the whole file, the method saves resources, saves the time of file transmission, and can further improve the working efficiency.
In the process of acquiring the file blocks by the user, the user firstly needs to register in a user information base, and after the registration, user information, file block set information and file access keys are sequentially verified to verify, and after all the verification is passed, the file blocks can be acquired, so that the safety of data can be ensured to the greatest extent.
Drawings
FIG. 1 is a schematic diagram of a file storage system according to the present invention.
FIG. 2 is a flowchart of the operation of the file storage system and file access terminal of the present invention.
Fig. 3 is a flow chart of checking the file compression code according to the present invention.
Detailed Description
The invention relates to a file storage and access system based on file cutting, which comprises a file storage system and a file access terminal,
the file storage system comprises a file block storage pool, a file information base, a user information base and a user key base, which are shown in figure 1; the file block storage pool is used for storing file blocks after file cutting; the file information base comprises a file basic attribute table and a file block detail table, wherein the file basic attribute table is used for recording file information before cutting, and the file block detail table is used for recording file block information after cutting; the user information base is used for registering the user accessing the file; the user key library is used for generating a file access key according to the file information library and the information stored in the user information library;
the file access terminal is used for sending a file compression code to the file storage system, and the file storage system sends file blocks to the file access terminal after the file compression code is checked by the file storage system.
Based on the system, the file storage and access method based on file cutting comprises the following steps:
storing the file information into a file basic attribute table of a file information base,
cutting the file, storing the cut file blocks into a file storage pool, and storing the cut file block information into a file block detail table of a file information base;
as shown in fig. 2, a user uploads a file compression code to a file storage system through a file access terminal, the file storage system checks the file compression code, and after the file compression code passes the check, the file storage system returns a corresponding file block to the file access terminal.
The file storage and access system based on file cutting mainly comprises 4 modules of a file block storage pool, a file information base, a user information base and a user key base. The file storage pool is used for storing file blocks, and is continuous or discontinuous storage space of a centralized or decentralized storage device, and each file block stored in the file block storage pool has a corresponding storage address.
The file information base is used for recording information of files and file blocks, and comprises a file basic attribute table and a file block detail table.
The file basic attribute table comprises a file identifier, a file name, a file type identifier, a file size identifier, a file identification code, and attribute fields, data types, functions and the like of the above attributes are shown in table 1:
table 1 file basic attribute table
| Sequence number | Attribute field | Attribute names | Data type |
| 1 | file_id | File identifier | Integer number |
| 2 | file_name | File name | Character(s) |
| 3 | file_type | File type identifier | Enumerating types |
| 4 | file_size | File size identifier | Integer number |
| 5 | file_code | File identification code | Character(s) |
The file identifier enables accurate finding of the file. The file type identifier is used for marking the type of the file and identifying the file as a picture, a text or a video. The file identification code is a character string calculated according to file data, the file identification codes of the same file are the same, and the same file can be ensured to be stored only once in a file storage system through inquiring and comparing the file identification codes.
The file block detail table is used for recording file block information after file cutting, and the file block marking detail table comprises file block identifiers used for marking file blocks; a file identifier for identifying a file to which the file block belongs, the file block size identifier for identifying the number of bytes of the file block; the file block sequence number is used for marking the sequence number of the file block, the serial value of the ith block is i, and the file block storage address identifier is used for identifying the storage address of the file block. The attribute fields, data types, functions, and the like of the above-described various attributes are shown in table 2.
TABLE 2 File block detail table
| Sequence number | Attribute field | Attribute names | Data type |
| 1 | block_id | File block identifier | Integer number |
| 2 | file_id | Belonging file identifier | Integer number |
| 3 | block_size | File block size identifier | Integer number |
| 4 | serial | File block sequence number | Integer number |
| 5 | address | File block storage address identifier | Character(s) |
The file storage system strictly controls access of files, and a user needs to have two necessary conditions for accessing a certain file: (1) Firstly, a user finishes registration in a user information base; (2) The user has a "file access key" for the access file.
The data type of "user details (info)" is not specifically defined herein, and may be a text object, an address pointing to a stored user information, etc., and the definition of the user information base is shown in table 3.
TABLE 3 user information base
| Sequence number | Attribute field | Attribute names | Data type |
| 1 | user_id | User identifier | Integer number |
| 2 | info | User detailed information | --- |
And the user key library is used for generating a file access key according to the file information library and the information stored in the user information library, and a user accesses a specified file through the file access key, and the definition of the user key library is shown in table 4.
Table 4 user key store
| Sequence number | Attribute field | Attribute names | Data type |
| 1 | user_id | User identifier | Integer number |
| 2 | file_id | File identifier | Integer number |
| 3 | file_key | File access key | Character(s) |
Examples
Now, suppose that the system needs to store a file with a file name "a" and a file size S, and the storage steps are as follows:
and 1, recording File information, wherein a File name file_name is denoted as File-A, and a file_size is denoted as S in a File basic attribute table.
And 2, cutting the file, wherein in the method for cutting the file in the embodiment, the file is equally cut according to the size of the file. The File "File-a" is cut, in this example, and assuming that the File is cut from scratch into n File blocks B1, B2, B3, bn of the same size, each File block is S/n in size. Bi (i > =1 and i < =n) represents the i-th file block after dicing.
Step 3, writing the file block information into a file block detail table, and enabling the file storage system to write the file block B into the file block detail table 1 、B 2 、B 3 、...、B n Stores, assuming that the storage addresses are A in turn 1 ，A 2 ，A 3 ，...，A n The file block identifiers are respectively T 1 ，T 2 ，T 3 ，....，T n The information of each file block is written in a file block detail table (block _ info), as shown in table 5,
TABLE 5 File Block information Table after File A cut
Assuming that the user named "U" needs to access the file, the user "U" is first used "Registering in a user information base with a user identifier of' Q U In order to allow the user "U" to have the right to access the file "A", a record is written in the user key store as follows, and the user_id is denoted as Q U File_id is denoted as F A The file_key is denoted as key (U, a), where key (U, a) is a function containing 2 parameters, the first parameter representing "user", the second parameter representing "file", the function taking as input "user" and "file", and then generating a key for the user to access the file.
The user 'U' accesses the file 'A' through the file access terminal, and the access process comprises two steps. Step 1: uploading a file compression code (compact_code) to a file storage system for a file access terminal; step 2: the file storage system returns one to a plurality of file blocks (blocks) to the file access terminal.
The file compression code consists of 4 parts including user information, file information, accessed file block set and file access key. In the above example, the user "U" may formally describe one of the file compression codes for file a as compression_code=u#a# { B 1 、B 2 、B 3 、...、B n And # key (U, a), where # is a separator. The compressed code indicates that the user 'U' needs to access the file 'A', and the accessed file block is { B } 1 、B 2 、B 3 、…、B n The set of file blocks in the file compression code specifies which file blocks the user needs to access, which set differs from file block to file block, for example when compression_code=u#a# { B 1 When the } # key (U, A), only the file block B is accessed 1 。
The verification process of the file storage system on the file compression code is shown in fig. 3, firstly, user information is verified, and whether the user is registered in the file storage system or not is verified; then checking the file information to check whether the file exists in the storage system; next, checking the file block set to check whether the file blocks exist in the file storage system and whether the file blocks are matched with file information in the file compression code, namely whether the file blocks belong to files specified in the file compression code; finally, the file access key information is checked to see if the key belongs to the user and the key of the file.
Claims (10)
1. The file storage and access system based on file cutting is characterized by comprising a file storage system and a file access terminal;
the file storage system comprises a file block storage pool, a file information base, a user information base and a user key base;
the file block storage pool is used for storing file blocks after file cutting; the file information base comprises a file basic attribute table and a file block detail table, wherein the file basic attribute table is used for recording file information before cutting, and the file block detail table is used for recording file block information after cutting; the user information base is used for registering the user accessing the file; the user key library is used for generating a file access key according to the file information library and the information stored in the user information library;
the file access terminal is used for sending a file compression code to the file storage system, and after the file compression code is checked by the file storage system, the file storage system sends a corresponding file block to the file access terminal according to the compression code.
2. The file storage and access system based on file cutting as claimed in claim 1, wherein the file information stored in the file base attribute table includes a file identifier, a file name, a file type identifier, a file size identifier, and a file identification code;
the file identifier is used for searching the corresponding file, the file type identifier is used for marking the type of the file, the file size identifier is used for identifying the byte number of the file, and the file identification code is used for distinguishing the corresponding file from the rest files.
3. The file storage and access system based on file cutting as claimed in claim 1, wherein the file block information stored in the file block list includes a file block identifier, a file block size identifier, a file block sequence number, and a file block storage address identifier;
the file block identifier is used for searching a corresponding file block; the file identifier is used for identifying the file to which the file block belongs, and the file block size identifier is used for identifying the byte number of the file block; the file block sequence number is used for marking the sequence number of the file block in the file block sequence, and the file block storage address identifier is used for identifying the storage address of the file block.
4. A file storage and access system based on file cutting as defined in claim 1 wherein said user information base includes a user identifier for recording the identity of the user.
5. The file storage and access system of claim 1 wherein the keystore further comprises a user identifier, a file identifier, and the file access key is a function of the user and the file.
6. The file storage and access system of any of claims 1-5, wherein the file storage pool is a plurality of non-contiguous storage spaces in a distributed storage device.
7. A file storage and access method based on file cutting, applied to the file storage and access system based on file cutting as claimed in any one of claims 1 to 6, characterized by comprising the following steps:
storing the file information into a file basic attribute table of a file information base;
cutting the file, storing the cut file blocks into a file storage pool, and storing the cut file block information into a file block detail table of a file information base;
the user uploads the file compression code to the file storage system through the file access terminal, the file storage system checks the file compression code, and after the file compression code passes the check, the file storage system returns the corresponding file block to the file access terminal.
8. The file storage and access method based on file cutting of claim 7, wherein the file compression code includes user information, file information, a set of accessed file blocks, and a file access key.
9. The file storage and access method based on file cutting as claimed in claim 8, wherein the file storage system checks the file compression code, including checking the user information, checking whether the user is registered in the user information base; checking the file information, and checking whether the file exists in a file information base; and checking the file block information, checking whether the file block exists in a file block storage pool, and checking whether the file block information in a file block detail table is matched with the file block information in the compression code, and checking a file access key, wherein the file access key belongs to the user and the file.
10. A method for storing and accessing files based on file cutting according to any of claims 7-9, wherein the method for cutting files comprises equally dividing files according to file size or cutting according to the definition of functional areas of data in files.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311113362.7A CN117171789A (en) | 2023-08-31 | 2023-08-31 | File storage and access system and method based on file cutting |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311113362.7A CN117171789A (en) | 2023-08-31 | 2023-08-31 | File storage and access system and method based on file cutting |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117171789A true CN117171789A (en) | 2023-12-05 |
Family
ID=88944185
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311113362.7A Pending CN117171789A (en) | 2023-08-31 | 2023-08-31 | File storage and access system and method based on file cutting |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117171789A (en) |
-
2023
- 2023-08-31 CN CN202311113362.7A patent/CN117171789A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210056217A1 (en) | Methods and apparatus to provide group-based row-level security for big data platforms | |
| US7664829B2 (en) | Document managing system, document managing apparatus and document managing method | |
| CN111177302B (en) | Service bill processing method, device, computer equipment and storage medium | |
| KR101270663B1 (en) | Method of encapsulating information in a database, an encapsulated database for use in a communication system and a method by which a database mediates an instant message in the system | |
| DE112005002955T5 (en) | Electrical transmission system in a secret environment between virtual disks and associated electrical transmission method | |
| TW201800967A (en) | Method and device for processing distributed streaming data | |
| CN111580965A (en) | Data request processing method and system | |
| WO2017174013A1 (en) | Data storage management method and apparatus, and data storage system | |
| CN111666205A (en) | Data auditing method, system, computer equipment and storage medium | |
| CN112685436A (en) | Traceability information processing method and device | |
| CN114925041A (en) | Data query method and device | |
| CN112912870A (en) | Tenant identifier conversion | |
| CN113010904A (en) | Data processing method and device and electronic equipment | |
| CN117171789A (en) | File storage and access system and method based on file cutting | |
| US20120089849A1 (en) | Cookie management system and method | |
| CN108874944B (en) | XSL language transformation-based heterogeneous data mapping system and method | |
| CN108228101B (en) | Method and system for managing data | |
| CN116910820A (en) | Data report processing method, device, computer equipment and storage medium | |
| CN116186116A (en) | Asset problem analysis method based on equal protection assessment | |
| US20060136361A1 (en) | Extensible, customizable database-driven row-level database security | |
| CN112416875A (en) | Log management method and device, computer equipment and storage medium | |
| CN108989245B (en) | User data storage method and device | |
| CN117171108B (en) | Virtual model mapping method and system | |
| CN116112264B (en) | Method and device for controlling access to strategy hidden big data based on blockchain | |
| US11928135B2 (en) | Edge computing data reproduction and filtering gatekeeper |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |