CN109492037B - Data acquisition method and device based on Redis and Logstash - Google Patents
Data acquisition method and device based on Redis and Logstash Download PDFInfo
- Publication number
- CN109492037B CN109492037B CN201811302690.0A CN201811302690A CN109492037B CN 109492037 B CN109492037 B CN 109492037B CN 201811302690 A CN201811302690 A CN 201811302690A CN 109492037 B CN109492037 B CN 109492037B
- Authority
- CN
- China
- Prior art keywords
- data
- file
- logstash
- redis
- acquisition method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention provides a data acquisition method based on Redis and Logstash, which comprises the following steps: the data receiving server receives data pushed by an external system; storing the pushed data into a file system through the Redis; the Logstash collects the data from the file system; and the Logstash converts the data and then sends the converted data to an application system. The data processing can be accelerated by utilizing a high-performance storage mechanism of Redis, and the stability of the service is improved and the error probability is reduced through decoupling operation of different services.
Description
Technical Field
The invention relates to the field of data acquisition and storage, in particular to a data acquisition method and equipment based on Redis and Logstash.
Background
In the open source data system, Redis, Memcached, and the like are commonly used. Redis is a key-value storage system, and compared with a distributed cache system Memcached, the key-value storage system supports relatively more value types to be stored, including string, list, set, zset, and hash. These data types all support push/pop, add/remove, and intersect union and difference, and richer operations, and these operations are all atomic. On this basis, Redis supports various different ways of ordering. Like Memcached, data is cached in memory to ensure efficiency. But the difference is that Redis periodically writes updated data to the disk or writes modification operations to an additional recording file, and realizes master-slave synchronization on the basis of the updated data or the modification operations. Therefore, Redis is a key-value database with higher performance, which greatly compensates the shortage of Memcached key/value storage and can well supplement a relational database in some occasions.
To extract data stored in Redis, as shown in FIGS. 1 and 2, a method is combined with Logstash, which is an open source data collection engine with real-time data transmission capability that can uniformly filter data from different sources and output to destinations according to the developer's specifications. The log analysis system is configured by combining the both. As shown in FIG. 1, deploying Logstash on all services needing log collection, as a Logstash-agent (also called as a Logstash wrapper) for monitoring and filtering the collected logs, sending the filtered contents to Redis to finish storage, and then collecting the logs together by using the Logstash-index1 and the Logstash-index2 in FIG. 1 and handing the logs to a full-text search service Cluster Elastic search-Cluster, and performing custom search by using the Elastic search-Cluster and performing page display by combining the custom search through Kibana, wherein a process of interaction between the Redis and the Logstash is shown in FIG. 2: (1) the external system pushes the data into the system, the system stores the data of the external system in Redis, and defines a uniform prefix key according to different data and the type of the stored data; (2) when the data are stored, setting the timeout time in the data values Redis, wherein the timeout time can be set to different values according to different service scenes and service requirements; (3) respectively storing the data of the part in a user-defined file in a ground mode by modifying the Redis source code; (4) a Logstash file reading plug-in (Logstash plug-in) monitors data in the incremental reading file; (5) defining different data cleaning and packaging filter plug-ins (Logstash plug-ins) according to different services and different structures of the acquired data, and loading and converting the data into a data format required by an external system; (6) according to different services and different storage media of external systems, different data outgoing plug-ins (Logstash self-contained plug-ins) are defined, and data are sent out and stored in different external systems.
Because the Redis needs to periodically write data into different disks in order to ensure the efficiency of the Redis, when the Redis is applied to data acquisition, a large amount of complex control logic needs to be written for the Redis because the data acquisition needs to store different data into different disks, so that the application complexity of the Redis is increased, and the phenomenon of performance reduction and instability of the Redis can be caused by higher application degree. The log analysis system utilizes Logstash to realize the data acquisition function of Redis, but the log analysis system is only limited to storage and acquisition of log data types, and the logic coupling of the log analysis system and the log data types is complex, so that the application of the log analysis system has great limitation.
Disclosure of Invention
In order to solve the technical problem of the Redis in data acquisition, although the Logstash and the Redis of a log monitoring and analyzing system are combined, the coupling degree between the Logstash and the Redis is reduced by innovatively designing the Logstash, and the method can adapt to different data types. In order to achieve the purpose, the invention provides a data acquisition and storage method and equipment based on Redis and Logstash. Specifically, the following technical scheme is adopted:
in a first aspect, the present invention provides a data acquisition method based on Redis and Logstash, which is characterized by comprising:
step 1, a data receiving server receives data pushed by an external system;
step 2, storing the pushed data into a file system through the Redis;
step 3, the Logstash collects the data from the file system;
and 4, converting the data by the Logstash and then sending the converted data to an application system.
Further, on the basis of the above technical solution, as a more preferable technical solution, the step 2 further includes:
and determining the service type of the received data and determining a file storage mode.
Further, on the basis of the above technical solution, as a more preferable technical solution, the step 2 further includes:
setting a first timer for data stored in a memory, storing the data in a file of a file system in a disk when the first timer exceeds a set first time threshold, and deleting the data in the memory;
and setting a second timer for the file stored in the file system in the disk, and marking the file when the second timer exceeds a set second duration threshold value to generate a marked file.
Further, on the basis of the above technical solution, as a more preferable technical solution, the step 3 further includes:
step 3.1, the Logstash comprises a plurality of reading plug-ins, and the plurality of reading plug-ins concurrently read the files in the file system;
3.2, when at least one reading plug-in is successfully read, successfully acquiring data, and taking the read file data as acquired data; otherwise, the data acquisition fails.
Further, on the basis of the above technical solution, as a more preferable technical solution, the step 4 further includes:
and if the data acquisition is successful, carrying out data cleaning and data encapsulation on the acquired data based on the data structure corresponding to the service type, converting the acquired data into a data format supported by the application system, and sending the data format to the application system through a corresponding forwarding plug-in.
Further, on the basis of the above technical solution, as a more preferable technical solution, the method further includes:
and periodically detecting whether the file system in the disk has the mark file, and if so, entering a mark file deleting process.
Further, on the basis of the above technical solution, as a more preferred technical solution, the process of deleting the markup file includes the following steps:
step a, judging whether the data in the mark file is sent to an application system, if so, turning to step d, otherwise, turning to step b;
step b, judging whether a second duration threshold corresponding to a second timer corresponding to the marker file is prolonged; if yes, turning to the step d, otherwise, turning to the step c;
c, extending the second time length threshold corresponding to the marking file by a set value, continuing timing of the second timer corresponding to the marking file, monitoring whether the timing of the second timer reaches the second time length threshold, if so, turning to the step a, otherwise, continuing to monitor the timing of the second timer;
and d, deleting the marker file.
Further, on the basis of the above technical solution, as a more preferable technical solution, the setting value is set as follows:
the set value is (free capacity of the disk × adjustment coefficient)/(average data write amount per unit time of the disk), and the adjustment coefficient is a positive number smaller than 1.
Further, on the basis of the above technical solution, as a more preferable technical solution, the adjustment coefficient is 0.625.
On the other hand, the invention also provides data acquisition equipment based on Redis and Logstash, which is characterized by comprising the following components:
a processor and a memory, the memory having a medium with program code stored therein, the electronic device being capable of performing the method of any of claims 1-9 when the processor reads the program code stored in the medium.
Compared with the prior art, the technical scheme of the embodiment of the invention has at least the following beneficial technical effects:
the method has the advantages that a high-performance storage mechanism of Redis is combined with an improved memory and disk data storage mechanism, data processing can be accelerated, service stability is improved and the probability of forced loss and errors of data is reduced through decoupling operation of different services, in addition, through concurrent reading of a plurality of plug-ins in the Logstash, the operation of writing codes to achieve data selection and conversion is reduced, and time cost and labor cost are saved.
Drawings
Fig. 1 and 2 are schematic structural diagrams of a log analysis system in the prior art;
fig. 3 and fig. 4 are schematic structural and flow diagrams of a preferred embodiment of the data acquisition method based on Redis and Logstash according to the present invention;
fig. 5 is a schematic diagram of a preferred embodiment of the data acquisition device based on Redis and Logstash according to the present invention.
The present invention is described in further detail below. The following examples are merely illustrative of the present invention and do not represent or limit the scope of the claims, which are defined by the claims.
Detailed Description
The technical scheme of the invention is further explained by the specific implementation mode in combination with the attached drawings.
To better illustrate the invention and to facilitate the understanding of the technical solutions thereof, typical but non-limiting examples of the invention are as follows: it should be noted that the embodiments listed in the description of the present invention are only exemplary embodiments for convenience of description, and should not be construed as the only correct embodiments of the present invention, and should not be construed as limiting the scope of the present invention.
Data acquisition and storage in big data applications is one of the most fundamental functions in big data applications. The method is characterized in that enough diversified data can be obtained as far as possible, so that compared with the traditional data acquisition and storage, the data acquisition and storage in the big data application has the distinct characteristics: firstly, when a large amount of data needs to be acquired, the data needs to be processed at a fast enough processing speed, secondly, a storage space with a limited capacity needs to be efficiently utilized, the phenomenon that the data is forced to be lost when the capacity is insufficient is avoided, and thirdly, the diversification of the data needs to have higher data adaptability so as to meet the processing of various types of data. Therefore, in big data applications, it is difficult to meet the application requirements by using the conventional technical solution of data sampling and storing.
For the convenience of understanding, the inventive concept and embodiments of the present invention are illustrated by the preferred embodiments, but the scope of the present invention is not limited to the embodiments.
Referring to fig. 3, the present embodiment includes 4 main parts: the system comprises a data receiving server for providing data receiving service, a Redis server for providing Redis service, a file system and a Logstash server for providing Logstash service. The data acquisition, storage and processing flow is as follows:
the data receiving service provided by the data receiving service end receives various data pushed by an external system;
after Redis receives data, the data is temporarily stored in a memory with a higher speed to meet the speed requirement of receiving big data, and meanwhile, the storage mode of the data is determined according to the service type of the received data, the service type of the data can be set according to the application requirement, and a better implementation mode is as follows: the service types are set according to the characteristics of the data sources, such as news service types taking news data pushed from a news portal website as a main body, advertisement service types taking advertisement data pushed from an advertisement portal website as a main body, and the like, and the data of the contents have different characteristics, such as that the news service types are updated quickly, the data volume is large, the advertisement types are updated relatively slowly, and the data repeatability is high. In order to ensure that data of different service classes can be collected and stored in time, different file storage modes are set according to the characteristics of the data of different service classes, for example, news service classes are stored in a compressed mode due to large data volume, and advertisement service classes are stored in a non-compressed mode. When the data in the memory meets certain requirements, the data is stored in a corresponding file of the file system,
for data which has been sent out in a file system or data which has been left for a long time, the data needs to be cleaned in time so as to reduce the occupation of the disk space. Therefore, the data receiving service also provides a timing scheduling mode to delete the expired file data, and the excessive use of the disk in the system is prevented. As another innovation, the invention provides a timer-based scheme to optimize the utilization of the data storage capacity of the memory and the file system. A more preferred embodiment is: setting a first timer for data stored in a memory, and when the first timer exceeds a set first time threshold, storing the data in a file of a file system in a disk, and deleting the data in the memory, thereby increasing the free space of the memory.
And setting a second timer for the file stored in the file system in the disk, and marking the file when the second timer exceeds a set second duration threshold value to generate a marked file. The markup file is processed in the following way:
and periodically detecting whether the file system in the disk has the mark file, and if so, entering a mark file deleting process. Further, the markup file deletion processing procedure comprises the following steps:
step a, judging whether the data in the mark file is sent to an application system, if so, turning to step d, otherwise, turning to step b;
step b, judging whether a second duration threshold corresponding to a second timer corresponding to the marker file is prolonged; if yes, turning to the step d, otherwise, turning to the step c;
c, extending the second time length threshold corresponding to the marking file by a set value, continuing timing of the second timer corresponding to the marking file, monitoring whether the timing of the second timer reaches the second time length threshold, if so, turning to the step a, otherwise, continuing to monitor the timing of the second timer;
and d, deleting the marker file.
The step c extends the second duration threshold as another innovative point of the present invention, and can effectively reduce the forced deletion of data due to the limitation of storage space. In a more preferred embodiment, the setting value is (free capacity of the disk × adjustment coefficient)/(average data write amount per unit time of the disk), and the adjustment coefficient is a positive number smaller than 1. Wherein the adjustment coefficient can be set to 0.625
A more preferred embodiment is: files are stored in units of days, and data overtime time is set, so that excessive data extrusion is prevented, and the situation that data is forced to be lost due to the unfavorable condition that the occupation of a memory and a disk is violently increased is avoided.
The Logstash end provides Logstash service, and the Logstash is an open source data collection engine with real-time pipeline processing capability. The read data is read in a plug-in mode, and invalid contents in the read data are filtered to generate structured data. In general, the Logstash reader plug-in is adapted to a data structure. However, in large data applications, due to the diversification of data, the forms of data structures required by the large data applications are very diverse, and the traditional processing mode is to arrange a plug-in for each type and then select from the plug-ins. However, the selection often requires judgment time, and under the condition that the received data volume of the data receiving server is large, the storage space of the Redis is more required, and if the space is made free in time, the selection has a very important meaning for receiving data in time. Therefore, if the Logstash server can take the data away as soon as possible, the Logstash server is beneficial to the utilization of the storage space. As another innovation point of the invention, the implementation mode comprises a plurality of file reading plug-ins, each reading plug-in converts the structured data in one format, the same data in the file system is read concurrently, and the structured data in at least one format is output, so that the time occupation caused by the data format selection operation is reduced. As a better implementation, a plurality of different data cleaning and packaging filter plug-ins can be defined according to different services and different structures of the collected data, and the data is converted into a data format required by an external system.
In addition, different data sending-out plug-ins are defined according to different services and different storage media of external systems, and the read data are sent out and stored in different external systems.
The invention utilizes the high-performance storage mechanism of Redis, can accelerate the processing of data, and improves the stability of service and reduces the error probability through the decoupling operation of different services. In addition, the configuration files of different Logstash plug-ins are innovatively and concurrently used, so that the operation of realizing data conversion by compiling codes is reduced, and the time cost and the labor cost are saved.
Referring to fig. 4, as another preferred embodiment, based on the above technical solution, the embodiment is specifically an example of collecting internet website access records and storing the access records into an ElasticSearch service.
Firstly, the Internet website sends the ticket record information of the website accessed by the user to the system, and the data is delivered to the data receiving service;
then, after the data receiving service receives the data, the data is stored in the Redis service;
then, the deleting file system scheduled at regular time for data receiving service stores the call ticket data exceeding the specified time limit, for example: the call ticket data of the previous 30 days;
then, the Redis service stores the data into a corresponding file system based on the data characteristics of the call ticket;
the Logstash reads file data in a file system, and after conversion, the data is sent out to a specified ElasticSearch service by using a corresponding plug-in, such as a Logstash-es-output plug-in;
and the user can retrieve the access data of the user and view the report form by using the high-performance data retrieval capability of the ElasticSearch service.
As a second aspect of the present invention, there is also provided an electronic apparatus, as shown in fig. 5, including: a processor 1120 and a memory 1140, the memory 1140 having a medium (computer-readable storage medium) storing program code, the electronic device being enabled to perform the above-mentioned method steps with the aid of the input unit 1130 and the output unit 1150 when the processor reads the program code stored in the medium.
For method implementation, the various embodiments presented herein may be implemented in a computer-readable medium using, for example, computer software, hardware, or any combination thereof. For a hardware implementation, various embodiments of the present invention may be implemented by using at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a processor, a microprocessor, an electronic unit designed to perform the functions described herein, and in some cases, various embodiments of the present invention may be implemented in the processor 1120 shown in the figure. For a software implementation, the various embodiments presented herein may be implemented with separate software modules that allow for performing at least one function or operation. The software codes may be implemented by software applications (or programs) written in any suitable programming language, which may be stored in memory 1140 and executed by processor 1120.
The applicant declares that the present invention illustrates the detailed structural features of the present invention through the above embodiments, but the present invention is not limited to the above detailed structural features, that is, it does not mean that the present invention must be implemented depending on the above detailed structural features. It should be understood by those skilled in the art that any modifications of the present invention, equivalent substitutions of selected components of the present invention, additions of auxiliary components, selection of specific modes, etc., are within the scope and disclosure of the present invention.
The preferred embodiments of the present invention have been described in detail, however, the present invention is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present invention within the technical idea of the present invention, and these simple modifications are within the protective scope of the present invention.
It should be noted that the various technical features described in the above embodiments can be combined in any suitable manner without contradiction, and the invention is not described in any way for the possible combinations in order to avoid unnecessary repetition.
In addition, any combination of the various embodiments of the present invention is also possible, and the same should be considered as the disclosure of the present invention as long as it does not depart from the spirit of the present invention.
Claims (10)
1. A data acquisition method based on Redis and Logstash is characterized by comprising the following steps:
step 1, a data receiving server receives data pushed by an external system;
step 2, storing the pushed data into a file system through the Redis;
step 3, the Logstash collects the data from the file system;
and 4, converting the data by the Logstash and then sending the converted data to an application system.
2. The data acquisition method of claim 1, wherein said step 2 further comprises:
and determining the service type of the received data and determining a file storage mode.
3. The acquisition method according to claim 2, wherein said step 2 further comprises:
setting a first timer for data stored in a memory, storing the data in a file of a file system in a disk when the first timer exceeds a set first time threshold, and deleting the data in the memory;
and setting a second timer for the file stored in the file system in the disk, and marking the file when the second timer exceeds a set second duration threshold value to generate a marked file.
4. The acquisition method according to claim 3, wherein said step 3 further comprises:
step 3.1, the Logstash comprises a plurality of reading plug-ins, and the plurality of reading plug-ins concurrently read the files in the file system;
3.2, when at least one reading plug-in is successfully read, successfully acquiring data, and taking the read file data as acquired data; otherwise, the data acquisition fails.
5. The acquisition method according to claim 4, wherein said step 4 further comprises:
and if the data acquisition is successful, carrying out data cleaning and data encapsulation on the acquired data based on the data structure corresponding to the service type, converting the acquired data into a data format supported by the application system, and sending the data format to the application system through a corresponding forwarding plug-in.
6. The acquisition method as set forth in claim 5, further comprising:
and periodically detecting whether the file system in the disk has the mark file, and if so, entering a mark file deleting process.
7. The acquisition method according to claim 6, wherein said markup file deletion process comprises the steps of:
step a, judging whether the data in the mark file is sent to an application system, if so, turning to step d, otherwise, turning to step b;
step b, judging whether a second duration threshold corresponding to a second timer corresponding to the marker file is prolonged; if yes, turning to the step d, otherwise, turning to the step c;
c, extending the second time length threshold corresponding to the marking file by a set value, continuing timing of the second timer corresponding to the marking file, monitoring whether the timing of the second timer reaches the second time length threshold, if so, turning to the step a, otherwise, continuing to monitor the timing of the second timer;
and d, deleting the marker file.
8. The acquisition method according to claim 7, characterized in that the set value is set as follows:
the set value is (free capacity of the disk × adjustment coefficient)/(average data write amount per unit time of the disk), and the adjustment coefficient is a positive number smaller than 1.
9. The acquisition method according to claim 8, characterized in that said adjustment factor is 0.625.
10. A data acquisition device based on Redis and Logstash, characterized by comprising:
a processor and a memory, the memory having a medium with program code stored thereon, the data acquisition device being capable of performing the method of any one of claims 1-9 when the processor reads the program code stored on the medium.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811302690.0A CN109492037B (en) | 2018-11-02 | 2018-11-02 | Data acquisition method and device based on Redis and Logstash |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811302690.0A CN109492037B (en) | 2018-11-02 | 2018-11-02 | Data acquisition method and device based on Redis and Logstash |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109492037A CN109492037A (en) | 2019-03-19 |
| CN109492037B true CN109492037B (en) | 2020-06-12 |
Family
ID=65692331
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811302690.0A Active CN109492037B (en) | 2018-11-02 | 2018-11-02 | Data acquisition method and device based on Redis and Logstash |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109492037B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111162920B (en) * | 2019-11-27 | 2021-10-08 | 中盈优创资讯科技有限公司 | Ticket processing method and device of Internet of things |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105243147A (en) * | 2015-10-22 | 2016-01-13 | 浪潮(北京)电子信息产业有限公司 | Slow query log management method and system of MySQL database |
| CN106130782A (en) * | 2016-07-19 | 2016-11-16 | 努比亚技术有限公司 | A kind of method and system obtaining server log |
| CN107577588A (en) * | 2017-09-26 | 2018-01-12 | 北京中安智达科技有限公司 | A kind of massive logs data intelligence operational system |
| CN107861859A (en) * | 2017-11-22 | 2018-03-30 | 北京汇通金财信息科技有限公司 | A kind of blog management method and system based on micro services framework |
| CN108376181A (en) * | 2018-04-24 | 2018-08-07 | 丹阳飓风物流股份有限公司 | Log services platform based on ELK |
| CN108712329A (en) * | 2018-05-02 | 2018-10-26 | 山东汇贸电子口岸有限公司 | A kind of gateway and log recording retrieval device based on Elasticsearch |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104536904B (en) * | 2014-12-29 | 2018-01-09 | 杭州华为数字技术有限公司 | A kind of method of data management, equipment and system |
| CN108629029B (en) * | 2018-05-09 | 2021-06-29 | 北京京东尚科信息技术有限公司 | Data processing method and device applied to data warehouse |
-
2018
- 2018-11-02 CN CN201811302690.0A patent/CN109492037B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105243147A (en) * | 2015-10-22 | 2016-01-13 | 浪潮(北京)电子信息产业有限公司 | Slow query log management method and system of MySQL database |
| CN106130782A (en) * | 2016-07-19 | 2016-11-16 | 努比亚技术有限公司 | A kind of method and system obtaining server log |
| CN107577588A (en) * | 2017-09-26 | 2018-01-12 | 北京中安智达科技有限公司 | A kind of massive logs data intelligence operational system |
| CN107861859A (en) * | 2017-11-22 | 2018-03-30 | 北京汇通金财信息科技有限公司 | A kind of blog management method and system based on micro services framework |
| CN108376181A (en) * | 2018-04-24 | 2018-08-07 | 丹阳飓风物流股份有限公司 | Log services platform based on ELK |
| CN108712329A (en) * | 2018-05-02 | 2018-10-26 | 山东汇贸电子口岸有限公司 | A kind of gateway and log recording retrieval device based on Elasticsearch |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109492037A (en) | 2019-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200175070A1 (en) | Low ram space, high-throughput persistent key-value store using secondary memory | |
| KR101224752B1 (en) | Efficient processing of time-bounded messages | |
| EP2965189B1 (en) | Managing operations on stored data units | |
| US8949189B2 (en) | Managing storage of individually accessible data units | |
| EP2965187B1 (en) | Managing operations on stored data units | |
| US11625412B2 (en) | Storing data items and identifying stored data items | |
| CN113391973B (en) | Internet of things cloud container log collection method and device | |
| AU2016394743A1 (en) | Method and apparatus for archiving database, and method and apparatus for searching archived database | |
| CN109492037B (en) | Data acquisition method and device based on Redis and Logstash | |
| CN111752804B (en) | Database cache system based on database log scanning | |
| Nath | Energy efficient sensor data logging with amnesic flash storage | |
| CN116881277A (en) | Data aggregation method, apparatus and computer readable medium | |
| CN116089364A (en) | Storage file management method and device, AI platform and storage medium | |
| CN108984720B (en) | Data query method and device based on column storage, server and storage medium | |
| Nath et al. | Online maintenance of very large random samples on flash storage | |
| CN114036121A (en) | Log file processing method, device, system, equipment and storage medium | |
| CN108874891B (en) | Method for facilitating reading of family cloud disk files | |
| US6487635B1 (en) | Management of orphan tracks | |
| CN111078643A (en) | Method and device for deleting files in batches and electronic equipment | |
| CN111782588A (en) | File reading method, device, equipment and medium | |
| US10037148B2 (en) | Facilitating reverse reading of sequentially stored, variable-length data | |
| CN113312194B (en) | Service data acquisition method and device | |
| KR101016320B1 (en) | Method of managing ubiquitous sensor network based real-time data and system | |
| CN113312353A (en) | Storage method and system for tracking journal | |
| JP3583814B2 (en) | file server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |