CN111930309A - File circular storage method and device based on data blocking technology - Google Patents
File circular storage method and device based on data blocking technology Download PDFInfo
- Publication number
- CN111930309A CN111930309A CN202010756855.2A CN202010756855A CN111930309A CN 111930309 A CN111930309 A CN 111930309A CN 202010756855 A CN202010756855 A CN 202010756855A CN 111930309 A CN111930309 A CN 111930309A
- Authority
- CN
- China
- Prior art keywords
- file
- block
- stored
- data
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000000903 blocking effect Effects 0.000 title claims abstract description 22
- 238000005516 engineering process Methods 0.000 title claims abstract description 22
- 238000004590 computer program Methods 0.000 claims description 17
- 238000013500 data storage Methods 0.000 claims description 7
- 125000004122 cyclic group Chemical group 0.000 description 7
- 238000000638 solvent extraction Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000011161 development Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Computer Security & Cryptography (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a file circular storage method and device based on a data blocking technology. The method comprises the following steps: data creation: creating the maximum file block number of the file and a memory corresponding to the file header in the storage area, wherein the file blocks comprise: the block head comprises a sequence number of a file block, a plurality of file blocks form a block chain, and data are stored in: the method comprises the steps of obtaining the data size of a file to be stored, determining the number of file blocks required for storing the file to be stored and the file pointer position corresponding to each file block according to the data size, writing the number of the data blocks and the file pointer position into a file header, and sequentially storing the file to be stored into block contents according to the memory occupied by the block contents and the sequence number. By adopting the method, the file can be circularly stored.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for circularly storing a file based on a data partitioning technology.
Background
With the development of computer technology, storage technology appears, and at the beginning of the development of computers, the storage requirement is not high, files are generally stored through a floppy disk, the size of the files is generally about 1M, and at present, with the increasing demand of file storage, the size of the files is also increased from small files of 1M to large files of more than 100M.
At present, storage modes for large files (more than 100M and less than 4G) are generally divided into the following two types:
1. the file mode with simple data arrangement generally adopts a mode of directly storing data in sequence to store the data into a file. For example, wav files, which are common in ordinary times, are a set of 4-byte single-precision floating-point data, which are sequentially written into the file. The storage and reading in the mode are simple and general, but the expansibility and the functionality are poor, and the recorded data is generally single, such as recording a section of sound, a section of current waveform and the like. More data cannot be recorded, and cyclic storage cannot be realized.
2. The multi-file approach generally requires one file index + several data files. For example, voltage data of a house at any moment needs to be stored, a general method is to generate an index file, generate a wav file every hour, record the voltage waveform of the hour, and write information of the path, the start time, the end time, the average value, the maximum value, the minimum value and the like of the wav file into the index file. The expansibility basically can meet most requirements, index files can be stored in data needing global storage, such as starting and ending time, and if circular storage is needed, the newly generated files replace the oldest files and the file names and the starting and ending time in the indexes are changed, so that circular storage can be realized. The defects are that maintenance management needs a certain technical level, at least two files need to be opened during writing and reading, and the writing and reading difficulty is high. And the multi-file mode makes the copy transfer action become complicated and unreliable, and the file is easy to lose and lose data.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a file circular storage method and apparatus based on a data partitioning technique, which can solve the problem of file circular storage difficulty.
A file circular storage method based on a data blocking technology comprises the following steps:
data creation:
creating a memory corresponding to the maximum file block number and the file header of the file in the storage area; the file block includes: chunk head and chunk content; the block header comprises a sequence number of the file block; the plurality of file blocks form a block chain;
data storage:
acquiring the size of data of a file to be stored, and determining the number of file blocks required for storing the file to be stored and the position of a file pointer corresponding to each file block according to the size of the data;
and writing the number of the data blocks and the position of the file pointer into the file header, and sequentially storing the files to be stored into the block contents according to the memory occupied by the block contents and the sequence number.
In one embodiment, the method further comprises the following steps: when the number of the file blocks is larger than the maximum number of the file blocks; and circularly storing the files to be stored into the block contents in sequence according to the memory occupied by the block contents and the serial numbers.
In one embodiment, the method further comprises the following steps: the file header is also used for recording a first starting time and a first ending time of the file to be stored; the block head is also used for recording a second starting time and a second ending time of the file to be stored in the block content; the second starting time to the second ending time are time segments from the first starting time to the first ending time.
In one embodiment, the method further comprises the following steps: and the file header and the file block occupy fixed contents.
In one embodiment, the method further comprises the following steps: and storing the file to be stored into the block content, and when the file to be stored in the block content is larger than the preset percentage of the memory, storing the file to be stored into the block content of the next file block according to the sequence number.
In one embodiment, the method further comprises the following steps: and when the file to be stored in the file block with the largest sequence number is larger than the preset percentage of the memory, storing the file to be stored in the block content of the file block with the smallest sequence number.
In one embodiment, the method further comprises the following steps: determining the file pointer position of the file block corresponding to the sequence number according to the sequence number; and reading the file to be stored in the file block according to the position of the file pointer.
A file circular storage apparatus based on data blocking technology, the apparatus comprising:
the creating module is used for creating the maximum file block number of the file and a memory corresponding to the file header in the storage area; the file block includes: chunk head and chunk content; the block header comprises a sequence number of the file block; the plurality of file blocks form a block chain;
the marking module is used for acquiring the data size of a file to be stored, and determining the number of file blocks required for storing the file to be stored and the position of a file pointer corresponding to each file block according to the data size;
and the storage module is used for writing the number of the data blocks and the positions of the file pointers into the file header and sequentially storing files to be stored into the block contents according to the memory occupied by the block contents and the sequence numbers.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
data creation:
creating a memory corresponding to the maximum file block number and the file header of the file in the storage area; the file block includes: chunk head and chunk content; the block header comprises a sequence number of the file block; the plurality of file blocks form a block chain;
data storage:
acquiring the size of data of a file to be stored, and determining the number of file blocks required for storing the file to be stored and the position of a file pointer corresponding to each file block according to the size of the data;
and writing the number of the data blocks and the position of the file pointer into the file header, and sequentially storing the files to be stored into the block contents according to the memory occupied by the block contents and the sequence number.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
data creation:
creating a memory corresponding to the maximum file block number and the file header of the file in the storage area; the file block includes: chunk head and chunk content; the block header comprises a sequence number of the file block; the plurality of file blocks form a block chain;
data storage:
acquiring the size of data of a file to be stored, and determining the number of file blocks required for storing the file to be stored and the position of a file pointer corresponding to each file block according to the size of the data;
and writing the number of the data blocks and the position of the file pointer into the file header, and sequentially storing the files to be stored into the block contents according to the memory occupied by the block contents and the sequence number.
According to the file cyclic storage method and device based on the data blocking technology, the computer equipment and the storage medium, the large files are stored into the plurality of file blocks in a blocking mode, indexing is carried out through the file headers and the block headers, file query is facilitated, when the files are stored, pressure on a magnetic disk can be reduced through blocking storage, the storage efficiency of software is improved, and data safety is improved. When data is in error, only part of files are lost, so that data loss is effectively prevented, and when all file blocks are fully stored during cyclic storage, the latest file covers the file in the oldest file block through a set strategy, so that the stored file still maintains the continuity of the file in time, and cyclic storage is realized.
Drawings
FIG. 1 is a block diagram of a file structure in one embodiment;
FIG. 2 is a flowchart illustrating a file circular storage method based on a data blocking technique according to an embodiment;
FIG. 3 is a block diagram of a file circular storage device based on data partitioning technology in one embodiment;
FIG. 4 is a diagram illustrating an internal structure of a computer device according to an embodiment;
fig. 5 is an internal structural view of a computer device in another embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The file circular storage method based on the data partitioning technology can be applied to terminals or servers. The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices, and the server can be implemented by an independent server or a server cluster formed by a plurality of servers.
Specifically, the terminal and the server both include a storage module, and when the method of the present application is executed, the file structure is as shown in fig. 1, and includes: the file structure comprises a file header and file blocks, wherein the file blocks comprise a block header and block contents, a plurality of blocks are arranged from top to bottom to form a block chain, and are aligned and sequenced according to positions of the blocks in the block chain, and sequence numbers are obtained.
In an embodiment, as shown in fig. 2, a file circular storage method based on a data partitioning technology is provided, which is described by taking an example that the method is applied to a terminal or a server, and includes the following steps:
data creation:
Firstly, a fixed-size storage location is determined in a storage area, a file block and a file header with the maximum number of file blocks are created in the storage location, and the file blocks include: chunk head and chunk content; the block header comprises the sequence number of the file block; multiple file blocks form a chain of blocks.
This step is a preparation stage of data storage, and it should be noted that if the terminal or the server already creates an existing storage location, data storage is directly performed without data creation.
The header can store various global information, such as: file block number, file block size, file type, start time, end time, and other relevant global parameters.
The block contents of a file block are used to store the data itself, and the block header is used to store the relevant parameters of the file block, such as block sequence number, block start time, block end time, data size and data type within the block.
Data storage:
and 204, acquiring the data size of the file to be stored, and determining the number of file blocks required for storing the file to be stored and the position of a file pointer corresponding to each file block according to the data size.
The file to be stored can be any size, can be stored once or stored in sequence, and can be data with fixed size or stored in sequence along with time, so that cyclic storage can be realized.
When the number of the file blocks exceeds the maximum number of the file blocks, the circular storage is realized. The file pointer location is an index for reading the file.
And step 206, writing the number of the data blocks and the position of the file pointer into a file header, and sequentially storing the file to be stored into the block contents according to the memory occupied by the block contents and the sequence number.
According to the file circulating storage method based on the data blocking technology, the large files are stored into the plurality of file blocks in a blocking mode, indexing is carried out through the file headers and the block headers, file query is facilitated, when the files are stored, pressure on a magnetic disk can be reduced through blocking storage, the disk storage efficiency of software is improved, and data safety is improved. When data is in error, only part of files are lost, so that data loss is effectively prevented, and when all file blocks are fully stored during cyclic storage, the latest file covers the file in the oldest file block through a set strategy, so that the stored file still maintains the continuity of the file in time, and cyclic storage is realized.
In one embodiment, when the number of the file blocks is greater than the maximum number of the file blocks, the file to be stored is cyclically stored in the block contents according to the memory occupied by the block contents and the sequence number. In this embodiment, when the number of file blocks is greater than the maximum number of file blocks, the file blocks need to be stored circularly.
In one embodiment, the file header is further configured to record a first start time and a first end time of the file to be stored, and the block header is further configured to record a second start time and a second end time of the file to be stored in the block content, where the second start time to the second end time are time slices from the first start time to the first end time. In this embodiment, the recorded files may be sequentially recorded files, so that a first initial time and a first end time for storing data are further recorded in the file header, and a second initial time to a second end time are also recorded in the block header, which is convenient for querying the files.
In one embodiment, the file header and file block occupy a fixed content.
In one embodiment, the file to be stored is stored in the block content, and when the file to be stored in the block content is larger than the preset percentage of the memory, the file to be stored is stored in the block content of the next file block according to the sequence number. The preset percentage may be determined according to actual conditions, for example: 90 percent.
In one embodiment, when the file to be stored in the file block with the largest sequence number stores more than a preset percentage of the memory, the file to be stored is stored in the block content of the file block with the smallest sequence number. In this embodiment, after the file block with the largest sequence number is stored fully, the file block with the smallest sequence number is circulated, so that the circular storage is realized.
In one embodiment, the method comprises the following steps:
a. the maximum number of blocks M of a file is specified when the file is created.
The maximum number of blocks may be determined according to the actual situation, for example, a maximum number of blocks of 1000 is created,
b. a memory of a header and a block of data is created and the size of each part is specified.
For example, in a specific scenario, the block header of a file block may be fixed to 64 bytes, the first byte of the block header is a block sequence number, the 0 th block sequence number is 0, and 100 bytes of the file header are written to the disk, where the write file pointer position is 0, and the file header must contain the maximum number of blocks M and the total number of blocks L.
c. When data is stored, firstly writing the data block into the memory, writing the 0 th block into the disk, writing the data block into the file pointer position, and preparing to write the 1 st block, wherein the serial number of the block is 1. And writing the total number L of blocks into the file header, wherein the total number L of blocks is 0.
If the size of the data block memory is larger than 90% of 5 Mbytes, the 0 th block is written into the disk. Since the header is set to 100 bytes, the pointer to the 0 th block write is 100.
d. When data is stored, firstly writing the memory of the data block, if the size of the memory of the data block is larger than 90% of 5 Mbytes, writing the 1 st block into a disk, and the pointer position of the write-in file is 100+5 x 1024 x 1. Ready to write block 2, with sequence number 2. And writing the total number L of blocks to the file header as 1.
5 x 1024 bytes equals 5M.
e. When data is stored, firstly writing the memory of the data block, if the size of the memory of the data block is larger than 90% of 5 Mbytes, writing the 2 nd block into a disk, and the pointer position of the write-in file is 100+5 x 1024 x 2. Block No. 3 is ready to be written, with sequence number 3. And writing the total number L of blocks into the file header to be 2.
f. And by analogy, when the nth block is written, the written file pointer position is 100+5 × 1024 × n. Then prepare to write the (n + 1) th block, with block number n + 1. And writing the total number L of blocks to the file header.
g. When the block sequence number n > is the maximum block number M. The write file pointer position is 100+5 1024 (n% M). Where n% M denotes the remainder of n dividing M, for example, 12% 1000 to 12, 1003% 1000 to 3, and 5018% 1000 to 18. Then prepare to write the (n + 1) th block, with block number n + 1. And writing the total number L of blocks into the file header. Therefore, the newly added block n covers the oldest block 0, and when the storage continues subsequently, the newly added block n +1 covers the block 1 with the longest storage time, so that the circular storage is realized.
When a file is read, the maximum block number M of the file needs to be acquired from a file header, and the whole file can be quickly traversed.
a. Opening the file, positioning the pointer position of the read file to be 0, reading 100 bytes, and obtaining the maximum block number M and the total block number L in the file header
b. The loop value s is from 0 to L-1 and the block header information of each block is read. The file read pointer for each block header is 100+5 1024 s.
c. According to the block sequence number information in the headers of all the blocks, the content in any one block can be read simply and conveniently. For example, when the content in block n is to be read, it is only necessary to locate the file read pointer to 100+5 × 1024 (n% M), and all the content of the file block with block number n can be obtained.
In summary, when the file circulation storage based on the data blocking technology is realized, compared with two modes of simple storage and multi-file storage in sequence, the file circulation storage based on the data blocking technology can realize single-file operation and can also take into account circulation storage and other functions of extended application information storage and the like. The file header can store the global information of the file, the file block can store the parameters local to the file block, and both the global information and the local parameters can be freely expanded, such as storing the name of a company, or the number of an order, or the number of a job, or the name of an operator, and the like. The method has the advantages that the method writes 5 Mbytes into a disk according to each block, compares simple storage and multi-file storage in sequence, enforces the size of the disk written into each time, and can reduce the pressure of the disk, improve the disk storage efficiency of software, improve the data security and prevent data loss compared with the mode that multiple small memory writing or super large memory writing is possibly caused without agreement. For example, 100 Mbytes, 10000 writes, 10000 bytes are written, the time consumed by ten times of test is 153,171,168,174,174,176,182,181,166,165, and the average total time is 171 milliseconds. The 100 Mbytes are written in 20 times according to a file circulation storage mode based on a data blocking technology, and the test time consumption is respectively 38,39,37,37,37,40,36,37,38 and 36 for ten times. The average total time taken is 37.5 milliseconds. It is seen that a small number of writes will reduce the speed of disk storage, and the disk also does not suggest a small memory write many times. A 100 mbyte write may be faster in time than a split of 20 writes, 5 mbytes per write, but 100 mbytes of data are lost in the event of an error or failure, 5 mbytes are written only, and only the current 5 mbytes of data are lost in the event of an unexpected error or failure.
It should be understood that, although the steps in the flowchart of fig. 2 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 2 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 3, there is provided a file circular storage apparatus based on a data partitioning technique, including: a creation module 302, a marking module 304, and a storage module 306, wherein:
a creating module 302, configured to create, in a storage area, the maximum file block number of a file and a memory corresponding to a file header; the file block includes: chunk head and chunk content; the block header comprises a sequence number of the file block; the plurality of file blocks form a block chain;
the marking module 304 is configured to obtain a data size of a file to be stored, and determine, according to the data size, the number of file blocks required to store the file to be stored and a file pointer position corresponding to each file block;
and the storage module 306 is configured to write the number of the data blocks and the position of the file pointer into the file header, and sequentially store files to be stored into the block contents according to the memory occupied by the block contents and the sequence number.
In one embodiment, the storage module 306 is further configured to, when the number of file blocks is greater than the maximum number of file blocks; and circularly storing the files to be stored into the block contents in sequence according to the memory occupied by the block contents and the serial numbers.
In one embodiment, the file header is further used for recording a first start time and a first end time of the file to be stored; the block head is also used for recording a second starting time and a second ending time of the file to be stored in the block content; the second starting time to the second ending time are time segments from the first starting time to the first ending time.
In one embodiment, the file header and the file block have fixed contents.
In one embodiment, the storage module 306 is further configured to store the to-be-stored file into the block content, and when the to-be-stored file stored in the block content is greater than a preset percentage of the memory, store the to-be-stored file into the block content of the next file block according to the sequence number.
In one embodiment, the storage module 306 is further configured to store the file to be stored in the block content of the file block with the smallest sequence number when the file to be stored in the file block with the largest sequence number stores more than a preset percentage of the memory.
In one embodiment, the reading module is configured to determine, according to the sequence number, a file pointer position of a file block corresponding to the sequence number; and reading the file to be stored in the file block according to the position of the file pointer.
For specific limitations of the file circulation storage apparatus based on the data blocking technology, reference may be made to the above limitations of the file circulation storage method based on the data blocking technology, and details are not repeated here. The modules in the file circulation storage device based on the data blocking technology can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 4. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for operating systems, computer programs, and storage areas required for block storage in the nonvolatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a file loop storage method based on a data blocking technique.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 5. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for operating systems, computer programs, and storage areas required for block storage in the nonvolatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a file loop storage method based on a data blocking technique. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the configurations shown in fig. 4 and 5 are merely block diagrams of some configurations relevant to the present disclosure, and do not constitute a limitation on the computing devices to which the present disclosure may be applied, and that a particular computing device may include more or less components than those shown, or combine certain components, or have a different arrangement of components.
In an embodiment, a computer device is provided, comprising a memory storing a computer program and a processor implementing the steps of the method in the above embodiments when the processor executes the computer program.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the method in the above-mentioned embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A file circular storage method based on a data blocking technology comprises the following steps:
data creation:
creating a memory corresponding to the maximum file block number and the file header of the file in the storage area; the file block includes: chunk head and chunk content; the block header comprises a sequence number of the file block; the plurality of file blocks form a block chain;
data storage:
acquiring the size of data of a file to be stored, and determining the number of file blocks required for storing the file to be stored and the position of a file pointer corresponding to each file block according to the size of the data;
and writing the number of the data blocks and the position of the file pointer into the file header, and sequentially storing the files to be stored into the block contents according to the memory occupied by the block contents and the sequence number.
2. The method of claim 1, further comprising:
when the number of the file blocks is larger than the maximum number of the file blocks;
and circularly storing the files to be stored into the block contents in sequence according to the memory occupied by the block contents and the serial numbers.
3. The method of claim 1, wherein the file header is further used for recording a first start time and a first end time of the file to be stored; the block head is also used for recording a second starting time and a second ending time of the file to be stored in the block content;
the second starting time to the second ending time are time segments from the first starting time to the first ending time.
4. A method according to any one of claims 1 to 3, wherein the header and block contents are fixed.
5. The method according to any one of claims 1 to 3, wherein sequentially storing the files to be stored into the block contents according to the memory occupied by the block contents and the sequence numbers comprises:
and storing the file to be stored into the block content, and when the file to be stored in the block content is larger than the preset percentage of the memory, storing the file to be stored into the block content of the next file block according to the sequence number.
6. The method of claim 2, wherein the cyclically storing the files to be stored into the block contents in sequence according to the memory occupied by the block contents and the sequence numbers comprises:
and when the file to be stored in the file block with the largest sequence number is larger than the preset percentage of the memory, storing the file to be stored in the block content of the file block with the smallest sequence number.
7. The method according to any one of claims 1 to 3, further comprising:
determining the file pointer position of the file block corresponding to the sequence number according to the sequence number;
and reading the file to be stored in the file block according to the position of the file pointer.
8. A file circulation storage device based on data blocking technology is characterized in that the device comprises:
the creating module is used for creating the maximum file block number of the file and a memory corresponding to the file header in the storage area; the file block includes: chunk head and chunk content; the block header comprises a sequence number of the file block; the plurality of file blocks form a block chain;
the marking module is used for acquiring the data size of a file to be stored, and determining the number of file blocks required for storing the file to be stored and the position of a file pointer corresponding to each file block according to the data size;
and the storage module is used for writing the number of the data blocks and the positions of the file pointers into the file header and sequentially storing files to be stored into the block contents according to the memory occupied by the block contents and the sequence numbers.
9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010756855.2A CN111930309A (en) | 2020-07-31 | 2020-07-31 | File circular storage method and device based on data blocking technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010756855.2A CN111930309A (en) | 2020-07-31 | 2020-07-31 | File circular storage method and device based on data blocking technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111930309A true CN111930309A (en) | 2020-11-13 |
Family
ID=73315909
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010756855.2A Pending CN111930309A (en) | 2020-07-31 | 2020-07-31 | File circular storage method and device based on data blocking technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111930309A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112579602A (en) * | 2020-12-22 | 2021-03-30 | 杭州趣链科技有限公司 | Multi-version data storage method and device, computer equipment and storage medium |
| CN113157211A (en) * | 2021-04-20 | 2021-07-23 | 武汉卓目科技有限公司 | Storage method for information record of embedded system |
| CN113609085A (en) * | 2021-08-18 | 2021-11-05 | 希尔塔(苏州)信息技术有限公司 | Automobile data rapid storage method |
| CN114217738A (en) * | 2021-11-25 | 2022-03-22 | 深圳益邦阳光有限公司 | Dynamic queue type cyclic storage method, device, equipment and medium |
| CN114237494A (en) * | 2021-11-23 | 2022-03-25 | 北京谊安医疗系统股份有限公司 | Data real-time storage method suitable for embedded equipment |
| WO2023029937A1 (en) * | 2021-08-31 | 2023-03-09 | 华为技术有限公司 | Data processing method, data processing apparatus, and related device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH03201147A (en) * | 1989-12-28 | 1991-09-03 | Fujitsu Ltd | Method for controlling data block of directly formed cyclic file |
| CN106940700A (en) * | 2016-01-05 | 2017-07-11 | 南京中兴新软件有限责任公司 | The covering write method of distributed file system, apparatus and system |
| CN111240599A (en) * | 2020-01-17 | 2020-06-05 | 北京马赫谷科技有限公司 | Data stream storage method and device |
-
2020
- 2020-07-31 CN CN202010756855.2A patent/CN111930309A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH03201147A (en) * | 1989-12-28 | 1991-09-03 | Fujitsu Ltd | Method for controlling data block of directly formed cyclic file |
| CN106940700A (en) * | 2016-01-05 | 2017-07-11 | 南京中兴新软件有限责任公司 | The covering write method of distributed file system, apparatus and system |
| CN111240599A (en) * | 2020-01-17 | 2020-06-05 | 北京马赫谷科技有限公司 | Data stream storage method and device |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112579602A (en) * | 2020-12-22 | 2021-03-30 | 杭州趣链科技有限公司 | Multi-version data storage method and device, computer equipment and storage medium |
| CN113157211A (en) * | 2021-04-20 | 2021-07-23 | 武汉卓目科技有限公司 | Storage method for information record of embedded system |
| CN113157211B (en) * | 2021-04-20 | 2022-11-18 | 武汉卓目科技有限公司 | Storage method for information record of embedded system |
| CN113609085A (en) * | 2021-08-18 | 2021-11-05 | 希尔塔(苏州)信息技术有限公司 | Automobile data rapid storage method |
| CN113609085B (en) * | 2021-08-18 | 2023-08-15 | 希尔塔(苏州)信息技术有限公司 | Automobile data quick storage method |
| WO2023029937A1 (en) * | 2021-08-31 | 2023-03-09 | 华为技术有限公司 | Data processing method, data processing apparatus, and related device |
| CN114237494A (en) * | 2021-11-23 | 2022-03-25 | 北京谊安医疗系统股份有限公司 | Data real-time storage method suitable for embedded equipment |
| CN114217738A (en) * | 2021-11-25 | 2022-03-22 | 深圳益邦阳光有限公司 | Dynamic queue type cyclic storage method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111930309A (en) | File circular storage method and device based on data blocking technology | |
| US8069317B2 (en) | Providing and utilizing high performance block storage metadata | |
| CN109597571B (en) | Data storage method, data reading method, data storage device, data reading device and computer equipment | |
| US11656952B2 (en) | Reliable key-value store with write-ahead-log-less mechanism | |
| US11227635B2 (en) | Recording device, readout device, recording method, recording program, readout method, readout program, and magnetic tape | |
| CN109614055B (en) | Snapshot creating method and device, electronic equipment and machine-readable storage medium | |
| CN111290919A (en) | Log file generation method and device, computer equipment and storage medium | |
| CN108829345B (en) | Data processing method of log file and terminal equipment | |
| CN112347002B (en) | flash data storage method, system, computer equipment and storage medium | |
| CN114924911B (en) | Method, device, equipment and storage medium for backing up effective data of Windows operating system | |
| CN110851162A (en) | SSD (solid State disk) firmware upgrading method and device compatible with RAID and non-RAID | |
| US20240143495A1 (en) | Method and apparatus for updating raid 5 check value, and medium | |
| US20060200656A1 (en) | Apparatus and method to capture data from an embedded device | |
| CN113312309B (en) | Snapshot chain management method, device and storage medium | |
| CN110471623B (en) | Hard disk file writing method, device, computer equipment and storage medium | |
| CN114217741A (en) | Storage method of storage device and storage device | |
| CN112579004A (en) | Method, system, computer equipment and storage medium for quickly deleting flash data | |
| CN111158609B (en) | Data storage method and device, computer equipment and storage medium | |
| CN115269274B (en) | Data recovery method, device, computer equipment and storage medium | |
| CN116501266B (en) | Message context processing method, device, computer equipment and storage medium | |
| CN114185620B (en) | Method and device for realizing acceleration of SSD firmware loading, computer equipment and storage medium | |
| CN115617580B (en) | Incremental backup and recovery method and system based on Shared SST (SST) file | |
| EP4033371A1 (en) | Hash based key value to block translation methods and systems | |
| US20210089403A1 (en) | Metadata table management scheme for database consistency | |
| CN115543215B (en) | Data writing operation and data reading operation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201113 |
|
| RJ01 | Rejection of invention patent application after publication |