CN112684983A - Data storage method and device, electronic equipment and readable storage medium - Google Patents
Data storage method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN112684983A CN112684983A CN202011590203.2A CN202011590203A CN112684983A CN 112684983 A CN112684983 A CN 112684983A CN 202011590203 A CN202011590203 A CN 202011590203A CN 112684983 A CN112684983 A CN 112684983A
- Authority
- CN
- China
- Prior art keywords
- data
- state
- target data
- storage
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000003860 storage Methods 0.000 title claims abstract description 151
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000013500 data storage Methods 0.000 title claims abstract description 27
- 230000008030 elimination Effects 0.000 claims description 13
- 238000003379 elimination reaction Methods 0.000 claims description 13
- 238000004519 manufacturing process Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 19
- 238000005516 engineering process Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 5
- 239000007787 solid Substances 0.000 description 5
- 230000001360 synchronised effect Effects 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 239000012634 fragment Substances 0.000 description 3
- 238000005192 partition Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000005034 decoration Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data storage method and device, electronic equipment and a readable storage medium. Wherein, the method comprises the following steps: determining the data state of the target data according to the first time when the target data is written into the storage server; and storing the target data to a storage position corresponding to the data state. The invention solves the technical problem that the real-time data reading speed is low because the caching strategy in the related technology can not effectively process the caching and reading process of the data and the real-time data is polluted by the historical data.
Description
Technical Field
The invention relates to the technical field of data storage, in particular to a data storage method, a data storage device, electronic equipment and a readable storage medium.
Background
The high-throughput distributed publish-subscribe messaging system Kafka assumes a unified data caching and distribution role in the overall data platform. Currently, there are multiple partitions (partitions) of multiple topics (Topic) on the same machine and the same disk. Under the background, the most common problem is that consumption (reading data) of different theme partitions on the same disk compete for I/O resources, and finally, a job with sufficient consumption resources is affected by a consumption job with insufficient resources due to resource competition, so that delay is generated and Service Level Agent (SLA) is affected.
The existing open source cache architecture scheme includes OpenCas and FlashCache, core design ideas of the two are similar, and SSD (Solid State Disk or Solid State Drive) and HDD (Hard Disk Drive/mechanical Hard Disk, Hard Disk Drive) are divided into a plurality of management units according to a fixed size by using a mode of distributing and routing an I/O resource request of a user by using a Device Mapping (DM) layer of an operating system, so that the SSD space and the HDD space are mapped. After the user request arrives, firstly, data is acquired from the SSD through the DM layer, if the data is not found, the data is read from the HDD, meanwhile, the data in the HDD is written back into the SSD, and if the SSD is insufficient in space, a part of SSD data is eliminated according to a Least Recently Used (LRU) strategy.
In the process of implementing the present application, the applicant finds that at least the following technical problems exist in the prior art:
1. due to the limited space of a cache (PageCache) in a storage hard disk, in the native Kafka, a job with insufficient consumption capability may pollute the cache data of the job with sufficient consumption capability, that is, the data reading is delayed due to the weak data reading capability, so that the data reading speed is slowed.
2. The open source scheme (OpenCas and FlashCache) realizes the SSD-based cache strategy in the kernel of the operating system, and can alleviate the core problem to a certain extent, but the core theory of the scheme is based on the 'data space locality principle' (data which is just accessed is likely to be accessed again), which is not completely consistent with the read-write characteristic of Kafka, so that the open source scheme cannot radically cure the problem.
3. Whether the original Kafka scheme or the open source caching scheme (OpenCas and FlashCache), if a Cache Miss (Cache Miss) occurs, data in the slow device is written back to the fast device, and the mechanism cannot avoid the phenomenon that Cache data of different operations are polluted mutually.
It can be seen that no effective solution to the above problems has been proposed in the related art.
Disclosure of Invention
Embodiments of the present invention provide a data storage method, an apparatus, an electronic device, and a readable storage medium, so as to at least solve the technical problem that a cache policy in the related art cannot effectively handle a caching and reading process of data, and real-time data is polluted by historical data, which results in a slow real-time data reading speed.
According to an aspect of an embodiment of the present invention, there is provided a data storage method, including: determining the data state of the target data according to the first time when the target data is written into a storage server; and storing the target data to a storage position corresponding to the data state.
Further, storing the target data to a storage location corresponding to the data state includes: when the data state is a writing state, storing the target data to a first memory; when the data state is a write completion state, synchronizing the target data to a second memory of the storage server; and when the data state is the elimination state, deleting the target data from the first memory.
Further, determining a data state of the target data according to a first time of writing of the target data into the storage server includes: determining that the target data is in a write state if the first time is less than a first time threshold; determining that the target data is in a write-complete state if the first time is greater than or equal to the first time threshold and less than a second time threshold; and determining that the target data is in an elimination state under the condition that the first time is greater than the second time threshold.
Further, before determining the data state of the target data according to the first time of writing the target data into the storage server, the method further includes: after receiving a production request acting on the storage server, storing first specified data corresponding to the production request to the first storage.
Further, after storing the first specific data corresponding to the production request to the first memory, the method further includes: receiving a consumption request acting on the storage server, wherein the consumption request is used for requesting second specified data; determining a designated data state corresponding to the second designated data; determining the storage position of the second designated data according to the designated data state; reading the second specified data based on the storage location.
Further, after storing the target data to the storage location corresponding to the data state, the method further includes: determining a backup position of backup data corresponding to the target data according to the storage position and a preset configuration file; and storing the backup data to the backup position.
According to another aspect of embodiments of the present invention, there is also provided a data storage device including: the determining unit is used for determining the data state of the target data according to the first time of writing the target data into the storage server; and the storage unit is used for storing the target data to a storage position corresponding to the data state.
Further, the storage unit includes: the storage module is used for storing the target data to a first memory when the data state is a writing state; the synchronization module is used for synchronizing the target data to a second memory of the storage server when the data state is a write completion state; and the processing module is used for deleting the target data from the first memory when the data state is the elimination state.
Further, the determining unit includes: a first determining module, configured to determine that the target data is in a write state when the first time is less than a first time threshold; a second determining module, configured to determine that the target data is in a write-complete state when the first time is greater than or equal to the first time threshold and is less than a second time threshold; a third determining module, configured to determine that the target data is in an elimination state when the first time is greater than the second time threshold.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including a processor, a memory, and a program or an instruction stored on the memory and executable on the processor, wherein the program or the instruction implements the steps of the data storage method as described above when executed by the processor.
According to another aspect of the embodiments of the present invention, there is also provided a readable storage medium on which a program or instructions are stored, the program or instructions, when executed by a processor, implementing the steps of the data storage method as described above.
In the embodiment of the invention, the data state of the target data is determined according to the first time when the target data is written into the storage server; and storing the target data to a storage position corresponding to the data state. And determining a corresponding data state according to the storage time of the target data, and then determining the storage position of the target data according to the data state, so that the real-time data is completely distributed in the solid-state memory, and the remote historical data is distributed in the mechanical memory. The method and the device avoid the pollution of the real-time data by the historical data, and further solve the technical problem that the reading speed of the real-time data is low because the caching strategy in the related technology cannot effectively process the caching and reading process of the data and the real-time data is polluted by the historical data.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a diagram illustrating an alternative hardware application scenario of a data storage method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart diagram of an alternative data storage method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an alternative data storage device according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Before describing the data storage method in this embodiment, an application scenario of the data storage method in this embodiment is first described. Fig. 1 is a schematic diagram of an optional hardware application scenario of the data storage method in this embodiment. The hardware application scene schematic diagram includes a storage server 10 and a terminal device 20, wherein the storage server 10 and the terminal device 20 can realize data transmission; the storage server 10 includes a first storage 110 and a second storage 120, and the first storage has a higher read/write speed than the second storage.
In a specific application scenario, the terminal device 20 continuously sends the target data to the storage server 10, and determines a data state of the target data according to a first time when the target data is written into the storage server 10; and storing the target data to a corresponding storage position in the data state storage server 10. Specifically, the storage to the first memory 110 or the second memory 120 may be determined according to the first time.
In the embodiment, the corresponding data state is determined according to the storage time of the target data, and then the storage position of the target data is determined according to the data state, so that the real-time data is completely distributed in the solid-state memory, and the remote historical data is distributed in the mechanical memory. The method and the device avoid the pollution of the real-time data by the historical data, and further solve the technical problem that the reading speed of the real-time data is low because the caching strategy in the related technology cannot effectively process the caching and reading process of the data and the real-time data is polluted by the historical data.
According to an embodiment of the present invention, there is provided a data storage method, as shown in fig. 2, the method including:
s202, determining the data state of the target data according to the first time when the target data is written into the storage server;
specifically, in this embodiment, the storage server receives target data sent from a preset terminal or a preset server at preset time intervals, for example, receives device log data from a service server. The target data comprises a plurality of data fragments, and the target data is stored in the storage server by sending the corresponding data fragments to the storage server at preset time intervals and finishing sending the data fragments in a plurality of time periods.
The first time when the target data is written to the storage server is counted from the time when the target data is written to the storage server, that is, the first data slice of the target data is written to the storage server.
The data states of the target data include, but are not limited to, a write state, a write complete state, and an eviction state. When the target data is in the writing process but the writing operation is not completed, the data state of the target data is the writing state. And after the target data is written, determining that the data state of the target data is a write-completed state. And after a period of time after the target data is written, determining that the target data is in a obsolete state.
And S204, storing the target data to a storage position corresponding to the data state.
The storage server in this embodiment is used to store data, and the physical storage device of the storage server includes, but is not limited to, a solid-state memory and a mechanical memory, where the read-write speed of the solid-state memory is greater than that of the mechanical memory. The storage device is used for storing recent real-time data in the solid-state memory, the time for writing the real-time data into the storage server is less than a preset time threshold, and the real-time data comprises but is not limited to target data being written into the storage server. And the historical data stored in the storage server for a long time are distributed in the mechanical memory, so that the corresponding speed of reading and writing the real-time service data is guaranteed to the greatest extent.
Therefore, when the target data is in a write state, the target data is stored in the solid-state memory, and when the target data is written, the target data is synchronized from the solid-state memory to the mechanical memory without requiring a real-time operation. And when the target data is in a deselected state, in order to avoid the target data from polluting the cache data in real-time operation, the target data is deleted from the solid-state memory, and only the target data in the mechanical memory is reserved.
It should be noted that, in this embodiment, the data state of the target data is determined according to the first time of writing the target data into the storage server; and storing the target data to a storage position corresponding to the data state. And determining a corresponding data state according to the storage time of the target data, and then determining the storage position of the target data according to the data state, so that the real-time data is completely distributed in the solid-state memory, and the remote historical data is distributed in the mechanical memory. The method and the device avoid the pollution of the real-time data by the historical data, and further solve the technical problem that the reading speed of the real-time data is low because the caching strategy in the related technology cannot effectively process the caching and reading process of the data and the real-time data is polluted by the historical data.
Optionally, in this embodiment, the storing the target data to the storage location corresponding to the data state includes, but is not limited to: when the data state is a writing state, storing the target data into a first memory; when the data state is a write completion state, synchronizing the target data to a second memory of the storage server; and when the data state is the elimination state, deleting the target data from the first memory.
Specifically, in the present embodiment, the data read/write speed of the first memory is greater than the data read/write speed of the second memory, the first memory includes, but is not limited to, a Solid State Disk (SSD), and the second memory includes, but is not limited to, a mechanical memory hdd (hard Disk Drive).
In a specific application scenario, when the target data starts to be written into the storage server and is not written completely, the target data is determined to be in a writing state, and then the target data is stored in the solid-state memory, so that the writing speed of the target data is guaranteed. And after the target data is written, under the condition that real-time operation is not needed, determining that the data state of the target data is a written state, and synchronizing the target data from the solid-state memory to the mechanical memory so as to realize the transition of the target data from the solid-state memory to the mechanical memory. And after a period of time after the target data is written, determining that the target data is in a obsolete state. And when the data state of the target data is the elimination state, deleting the target data from the solid-state memory and only keeping the target data in the mechanical memory in order to avoid the target data from polluting the cache data in real-time operation.
Through the embodiment, the storage position of the target data is determined according to the data state, so that the real-time data is completely distributed in the solid-state memory, and the remote historical data is distributed in the mechanical memory. Real-time data is prevented from being polluted by historical data.
Optionally, in this embodiment, the determining the data state of the target data according to the first time of writing the target data into the storage server includes, but is not limited to: determining that the target data is in a writing state when the first time is less than a first time threshold; determining that the target data is in a write-in completion state when the first time is greater than or equal to a first time threshold and less than a second time threshold; and under the condition that the first time is greater than the second time threshold, determining that the target data is in an elimination state.
Specifically, determining the data state of the target data according to the first time when the target data is written into the storage server mainly includes writing the target data into the storage server when the first time is less than a first time threshold, determining that the target data is in a write state, and determining that the writing of the target data fails and the storage server reports an error if the target data is not written after the first time threshold is exceeded.
Next, in the event that the first time is greater than or equal to a first time threshold and less than a second time threshold, the target data is synchronized from the solid-state memory to the mechanical memory during the time period. Similarly, after the second time threshold is exceeded, if the target data does not complete the synchronization to the mechanical storage, it is determined that the target data synchronization fails, and the storage server reports an error.
And then, in the case that the first time is greater than the second time threshold, the target data is synchronized to the mechanical memory at the moment, and the time length of the target data in the storage server is relatively long, the target data is determined to be in a knockout state.
It should be noted that, in this embodiment, the first time threshold and the second time threshold may be set according to practical experience, and this embodiment is not limited in any way.
Through the embodiment, the data state of the target data is determined based on the first time when the target data is written into the storage server, so that the real-time data and the historical data of the real-time service are distinguished.
Optionally, in this embodiment, before determining the data state of the target data according to the first time of writing the target data into the storage server, the method further includes, but is not limited to: after receiving a production request acting on the storage server, storing first specified data corresponding to the production request to a first storage.
Specifically, in this embodiment, the production request acting on the storage server is sent by the preset terminal or the preset service server to the storage server, and is used for requesting to store the first specified data in the storage server. The first specified data is stored in the first storage at the storage server in response to the production request. The first memory is a solid state memory SSD. The characteristic of high data reading and writing speed of the solid-state memory is utilized to realize the rapid implementation of real-time services.
Optionally, in this embodiment, after storing the first specific data corresponding to the production request in the first storage, the method further includes, but is not limited to: receiving a consumption request acting on the storage server, wherein the consumption request is used for requesting second specified data; determining a designated data state corresponding to the second designated data; determining a storage position of second specified data according to the specified data state; the second specifying data is read based on the storage location.
Specifically, in the present embodiment, the consumption request acting on the storage server is used to request the storage server to read the second designated data, after the authentication is successful, the designated data state of the requested second designated data is determined, the storage location of the second designated data is determined according to the designated data state, and the second designated data is read based on the determined storage location. In one example, a storage server receives a consumption request for requesting to read log data, and if the specified data state of the log data is a write-in state, the storage position of the log data is determined to be in a solid-state memory; if the designated data state of the designated data is a write completion state, the log data is stored in the solid-state memory and the mechanical memory from the storage location.
Through the embodiment, the second specified data is read from the storage position where the second specified data is determined according to the specified data state, so that the real-time data is completely distributed in the solid-state storage, and the remote historical data is distributed in the mechanical storage, and the real-time data is prevented from being polluted by the historical data.
Optionally, in this embodiment, after storing the target data to the storage location corresponding to the data state, the method further includes, but is not limited to: determining a backup position of backup data corresponding to the target data according to the storage position and a preset configuration file; and storing the backup data to the backup position.
In a specific application scenario, for backup data of target data, after the target data is stored to a storage location corresponding to a data state, a backup location of the backup data is determined according to the storage location of the target data and a preset configuration file corresponding to the backup data, and the backup data is stored to the corresponding backup location. In one example, data is synchronized from the target data to the backup storage server in real time during the backup process of the target data, and the synchronized data can be determined to be written into the solid state memory SSD or the mechanical memory HDD by a configuration item in a preset configuration file.
According to the embodiment, the backup position of the backup data corresponding to the target data is determined according to the storage position and the preset configuration file, and the backup of the target data is completed in the process of adjusting the storage position of the target data according to the data state, so that the safety of the data is enhanced.
According to the embodiment of the invention, the data state of the target data is determined according to the first time when the target data is written into the storage server; and storing the target data to a storage position corresponding to the data state. And determining a corresponding data state according to the storage time of the target data, and then determining the storage position of the target data according to the data state, so that the real-time data is completely distributed in the solid-state memory, and the remote historical data is distributed in the mechanical memory. The method and the device avoid the pollution of the real-time data by the historical data, and further solve the technical problem that the reading speed of the real-time data is low because the caching strategy in the related technology cannot effectively process the caching and reading process of the data and the real-time data is polluted by the historical data.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
Example 2
According to an embodiment of the present invention, there is also provided a data storage apparatus for implementing the data storage method, as shown in fig. 3, the apparatus including:
1) a determining unit 30, configured to determine a data state of the target data according to a first time when the target data is written into a storage server;
2) and the storage unit 32 is used for storing the target data to a storage position corresponding to the data state.
Optionally, in this embodiment, the storage unit 32 includes:
1) the storage module is used for storing the target data to a first memory when the data state is a writing state;
2) the synchronization module is used for synchronizing the target data to a second memory of the storage server when the data state is a write completion state;
3) and the processing module is used for deleting the target data from the first memory when the data state is the elimination state.
Optionally, in this embodiment, the determining unit 30 includes:
1) a first determining module, configured to determine that the target data is in a write state when the first time is less than a first time threshold;
2) a second determining module, configured to determine that the target data is in a write-complete state when the first time is greater than or equal to the first time threshold and is less than a second time threshold;
3) a third determining module, configured to determine that the target data is in an elimination state when the first time is greater than the second time threshold.
By the data storage device provided by the embodiment, the data state of the target data is determined according to the first time when the target data is written into the storage server; and storing the target data to a storage position corresponding to the data state. And determining a corresponding data state according to the storage time of the target data, and then determining the storage position of the target data according to the data state, so that the real-time data is completely distributed in the solid-state memory, and the remote historical data is distributed in the mechanical memory. The method and the device avoid the pollution of the real-time data by the historical data, and further solve the technical problem that the reading speed of the real-time data is low because the caching strategy in the related technology cannot effectively process the caching and reading process of the data and the real-time data is polluted by the historical data.
Example 3
There is also provided, according to an embodiment of the present invention, an electronic device for implementing the data storage method, including a processor, a memory, and a program or instructions stored on the memory and executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the data storage method according to embodiment 1.
Optionally, in this embodiment, the memory is configured to store program code for performing the following steps:
s1, determining the data state of the target data according to the first time when the target data is written into the storage server;
and S2, storing the target data to a storage position corresponding to the data state.
Optionally, the specific example in this embodiment may refer to the example described in embodiment 1 above, and this embodiment is not described again here.
Example 4
Embodiments of the present invention also provide a readable storage medium on which a program or instructions are stored, which when executed by a processor implement the steps of the data storage method according to embodiment 1.
Optionally, in this embodiment, the readable storage medium is configured to store program code for performing the following steps:
s1, determining the data state of the target data according to the first time when the target data is written into the storage server;
and S2, storing the target data to a storage position corresponding to the data state.
Optionally, the storage medium is further configured to store program codes for executing the steps included in the method in embodiment 1, which is not described in detail in this embodiment.
Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
Optionally, the specific example in this embodiment may refer to the example described in embodiment 1 above, and this embodiment is not described again here.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing one or more computer devices (which may be personal computers, servers, network devices, etc.) to execute all or part of the steps of the method according to the embodiments of the present invention.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (11)
1. A method of storing data, comprising:
determining the data state of the target data according to the first time when the target data is written into a storage server;
and storing the target data to a storage position corresponding to the data state.
2. The method of claim 1, wherein storing the target data to a storage location corresponding to the data state comprises:
when the data state is a writing state, storing the target data to a first memory;
when the data state is a write completion state, synchronizing the target data to a second memory of the storage server;
and when the data state is the elimination state, deleting the target data from the first memory.
3. The method of claim 2, wherein determining the data state of the target data based on the first time the target data was written to the storage server comprises:
determining that the target data is in a write state if the first time is less than a first time threshold;
determining that the target data is in a write-complete state if the first time is greater than or equal to the first time threshold and less than a second time threshold;
and determining that the target data is in an elimination state under the condition that the first time is greater than the second time threshold.
4. The method of claim 1, further comprising, prior to determining the data state of the target data based on the first time of writing of the target data into the storage server:
after receiving a production request acting on the storage server, storing first specified data corresponding to the production request to the first storage.
5. The method of claim 4, further comprising, after storing the first specified data corresponding to the production request to the first memory:
receiving a consumption request acting on the storage server, wherein the consumption request is used for requesting second specified data;
determining a designated data state corresponding to the second designated data;
determining the storage position of the second designated data according to the designated data state;
reading the second specified data based on the storage location.
6. The method of claim 1, further comprising, after storing the target data to the storage location corresponding to the data state:
determining a backup position of backup data corresponding to the target data according to the storage position and a preset configuration file;
and storing the backup data to the backup position.
7. A data storage device, comprising:
the determining unit is used for determining the data state of the target data according to the first time of writing the target data into the storage server;
and the storage unit is used for storing the target data to a storage position corresponding to the data state.
8. The apparatus of claim 7, wherein the storage unit comprises:
the storage module is used for storing the target data to a first memory when the data state is a writing state;
the synchronization module is used for synchronizing the target data to a second memory of the storage server when the data state is a write completion state;
and the processing module is used for deleting the target data from the first memory when the data state is the elimination state.
9. The apparatus of claim 8, wherein the determining unit comprises:
a first determining module, configured to determine that the target data is in a write state when the first time is less than a first time threshold;
a second determining module, configured to determine that the target data is in a write-complete state when the first time is greater than or equal to the first time threshold and is less than a second time threshold;
a third determining module, configured to determine that the target data is in an elimination state when the first time is greater than the second time threshold.
10. An electronic device comprising a processor, a memory, and a program or instructions stored on the memory and executable on the processor, the program or instructions when executed by the processor implementing the steps of the data storage method of claims 1-6.
11. A readable storage medium, characterized in that it stores thereon a program or instructions which, when executed by a processor, implement the steps of the data storage method according to claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011590203.2A CN112684983A (en) | 2020-12-28 | 2020-12-28 | Data storage method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011590203.2A CN112684983A (en) | 2020-12-28 | 2020-12-28 | Data storage method and device, electronic equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112684983A true CN112684983A (en) | 2021-04-20 |
Family
ID=75454871
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011590203.2A Withdrawn CN112684983A (en) | 2020-12-28 | 2020-12-28 | Data storage method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112684983A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH103434A (en) * | 1996-06-14 | 1998-01-06 | Nec Corp | Semiconductor disk device and its write system |
| US20100100664A1 (en) * | 2008-10-21 | 2010-04-22 | Hitachi, Ltd. | Storage system |
| CN103019969A (en) * | 2011-09-27 | 2013-04-03 | 威刚科技（苏州）有限公司 | Flash memory storage device and determination method of bad storage area thereof |
| CN104035730A (en) * | 2014-06-04 | 2014-09-10 | 深圳市江波龙电子有限公司 | File write-in method, device and wireless storage equipment |
| CN106844236A (en) * | 2016-12-27 | 2017-06-13 | 北京五八信息技术有限公司 | The date storage method and device of terminal device |
| CN108427539A (en) * | 2018-03-15 | 2018-08-21 | 深信服科技股份有限公司 | Offline duplicate removal compression method, device and the readable storage medium storing program for executing of buffer memory device data |
| CN111309720A (en) * | 2018-12-11 | 2020-06-19 | 北京京东尚科信息技术有限公司 | Time sequence data storage method, time sequence data reading method, time sequence data storage device, time sequence data reading device, electronic equipment and storage medium |
| CN111367755A (en) * | 2020-02-17 | 2020-07-03 | 上海基分文化传播有限公司 | User log writing method and system of mobile terminal |
| CN111782134A (en) * | 2019-06-14 | 2020-10-16 | 北京京东尚科信息技术有限公司 | Data processing method, device, system and computer readable storage medium |
-
2020
- 2020-12-28 CN CN202011590203.2A patent/CN112684983A/en not_active Withdrawn
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH103434A (en) * | 1996-06-14 | 1998-01-06 | Nec Corp | Semiconductor disk device and its write system |
| US20100100664A1 (en) * | 2008-10-21 | 2010-04-22 | Hitachi, Ltd. | Storage system |
| CN103019969A (en) * | 2011-09-27 | 2013-04-03 | 威刚科技（苏州）有限公司 | Flash memory storage device and determination method of bad storage area thereof |
| CN104035730A (en) * | 2014-06-04 | 2014-09-10 | 深圳市江波龙电子有限公司 | File write-in method, device and wireless storage equipment |
| CN106844236A (en) * | 2016-12-27 | 2017-06-13 | 北京五八信息技术有限公司 | The date storage method and device of terminal device |
| CN108427539A (en) * | 2018-03-15 | 2018-08-21 | 深信服科技股份有限公司 | Offline duplicate removal compression method, device and the readable storage medium storing program for executing of buffer memory device data |
| CN111309720A (en) * | 2018-12-11 | 2020-06-19 | 北京京东尚科信息技术有限公司 | Time sequence data storage method, time sequence data reading method, time sequence data storage device, time sequence data reading device, electronic equipment and storage medium |
| CN111782134A (en) * | 2019-06-14 | 2020-10-16 | 北京京东尚科信息技术有限公司 | Data processing method, device, system and computer readable storage medium |
| CN111367755A (en) * | 2020-02-17 | 2020-07-03 | 上海基分文化传播有限公司 | User log writing method and system of mobile terminal |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107526546B (en) | Spark distributed computing data processing method and system | |
| US11561930B2 (en) | Independent evictions from datastore accelerator fleet nodes | |
| CN103593147B (en) | A kind of method and device of digital independent | |
| CN102629941B (en) | Caching method of a virtual machine mirror image in cloud computing system | |
| CN102782670B (en) | Memory cache data center | |
| US20150089121A1 (en) | Managing A Cache On Storage Devices Supporting Compression | |
| US11093410B2 (en) | Cache management method, storage system and computer program product | |
| CN106649349A (en) | Method, device and system for data caching, applicable to game application | |
| CN105677580A (en) | Method and device for accessing cache | |
| CN103812849A (en) | Local cache updating method and system, client and server | |
| CN107197359B (en) | Video file caching method and device | |
| CN102541984A (en) | File system of distributed type file system client side | |
| CN103152390A (en) | Method and device and nodes and system for node configuration of distributed storage system | |
| CN107368608A (en) | The HDFS small documents buffer memory management methods of algorithm are replaced based on ARC | |
| CN109471843A (en) | A kind of metadata cache method, system and relevant apparatus | |
| CN111930305A (en) | Data storage method and device, storage medium and electronic device | |
| CN114327280A (en) | Message storage method and system based on cold-hot separation storage | |
| CN104408126B (en) | A kind of persistence wiring method of database, device and system | |
| CN106326143B (en) | A kind of caching distribution, data access, data transmission method for uplink, processor and system | |
| CN113742131A (en) | Method, electronic device and computer program product for storage management | |
| CN109977074B (en) | HDFS-based LOB data processing method and device | |
| CN111831221A (en) | Distributed storage method and system based on cloud storage | |
| CN112684983A (en) | Data storage method and device, electronic equipment and readable storage medium | |
| CN101819589B (en) | Method and device for controlling file to be input into/output from cache | |
| US20070022251A1 (en) | Data caching method and computer program product therefor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210420 |