WO2021031640A1 - 数据读写方法、计算机设备与计算机可读存储介质 - Google Patents

数据读写方法、计算机设备与计算机可读存储介质 Download PDF

Info

Publication number
WO2021031640A1
WO2021031640A1 PCT/CN2020/091814 CN2020091814W WO2021031640A1 WO 2021031640 A1 WO2021031640 A1 WO 2021031640A1 CN 2020091814 W CN2020091814 W CN 2020091814W WO 2021031640 A1 WO2021031640 A1 WO 2021031640A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
message header
file
entity
reading
Prior art date
Application number
PCT/CN2020/091814
Other languages
English (en)
French (fr)
Inventor
钱勇
Original Assignee
深圳中兴网信科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳中兴网信科技有限公司 filed Critical 深圳中兴网信科技有限公司
Publication of WO2021031640A1 publication Critical patent/WO2021031640A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G06F3/0611—Improving I/O performance in relation to response time
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/062—Securing storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system

Definitions

  • This application relates to the field of data storage technology, such as a method for reading and writing data, a computer device, and a computer-readable storage medium.
  • the embodiment of the first aspect of the present application proposes a data reading and writing method with fast reading and writing.
  • the embodiment of the second aspect of the present application proposes a computer device.
  • the embodiment of the third aspect of the present application proposes a computer-readable storage medium.
  • the present application proposes a data reading and writing method, including: splitting a data object into a message header and a data entity; storing header tag information and the number of fields of the data entity in the message header And field information; store the data row in the data entity, add tag information to the data row; store the message header and data entity to a local file.
  • the present application proposes a computer device, including: a processor and a memory, and a computer program is stored in the memory.
  • a computer program is stored in the memory.
  • the computer program When the computer program is executed by the processor, it implements any of the above technical solutions.
  • the present application proposes a computer storage medium with a computer program stored on the computer storage medium.
  • the computer program is executed, the data read as described in any of the above technical solutions is realized. Writing method.
  • FIG. 1 shows a flowchart of a data reading and writing method provided by the first embodiment of the present application
  • FIG. 2 shows a flowchart of a data reading and writing method provided by a second embodiment of the present application
  • FIG. 3 shows a flowchart of a data reading and writing method provided by a third embodiment of the present application
  • FIG. 4 shows a schematic diagram of the storage structure of a data file in the data reading and writing method provided by the third embodiment of the present application.
  • Fig. 1 shows a flowchart of a data reading and writing method provided by the first embodiment of the present application.
  • the flow of the data reading and writing method provided by the first embodiment of the present application includes the following steps:
  • Step 102 Split the data object into a message header and data entities.
  • Step 104 Store the header tag information, the number of fields of the data entity, and the field information in the message header.
  • Step 106 Store the data row in the data entity, and add mark information to the data row.
  • Step 108 Store the message header and data entity to a local file.
  • the data reading and writing method proposed in this application splits the data file into a message header and a data entity, and establishes an association relationship between the message header and the data entity through the mark of a private header, and then directly stores the message header and the data entity locally In the file, when reading the file, you can directly search for the message header, and then read the data entity corresponding to the message header, and the method of directly storing to the local file does not need to consider the installation of the database, because the limitation of the database is cancelled, so , Both reading and writing data files have faster speed.
  • Fig. 2 shows a flow chart of the data reading and writing method provided by the second embodiment of the present application.
  • the process of the data reading and writing method provided by the second embodiment of the present application includes the following steps:
  • Step 202 Split the data object into a message header and data entities.
  • Step 204 Store the header tag information, the number of fields of the data entity, and the field information in the message header.
  • Step 206 Store the data row in the data entity.
  • Step 208 Judge whether the data row has data information; if the judgement is no, execute step 210, and if the judgement result is yes, execute step 212.
  • Step 210 Add a blank data mark to the data row.
  • Step 212 Add a value data mark to the data row.
  • Step 214 Encrypt the message header and the data entity to form a binary file.
  • Step 216 Store the binary file as a data file.
  • the data reading and writing method proposed in this application splits the data file into a message header and a data entity, and establishes an association relationship between the message header and the data entity through the mark of a private header, and then directly stores the message header and the data entity locally In the file, when reading the file, you can directly search for the message header, and then read the data entity corresponding to the message header, and the method of directly storing to the local file does not need to consider the installation of the database, because the limitation of the database is cancelled, so , Both reading and writing data files have faster speed.
  • each data row is marked to ensure data integrity.
  • the message header and the data entity are encrypted with the private key to obtain and store the binary message header and data entity.
  • the binary file occupies a small space and can be decrypted quickly, while improving the security of the data file.
  • storing the message header and the data entity as one data file can ensure that when multiple data objects are stored, multiple data files do not interfere with each other.
  • Fig. 3 shows a flowchart of a data reading and writing method provided by a third embodiment of the present application.
  • the process of the data reading and writing method provided by the third embodiment of the present application includes the following steps:
  • Step 302 Split the data object into a message header and data entities.
  • Step 304 Store header tag information, the number of fields of the data entity, and field information in the message header.
  • Step 306 Store the data row in the data entity.
  • Step 308 Determine whether the data row has data information; in the case where the judgment result is yes, execute step 312, and if the judgment result is no, execute step 310.
  • Step 310 Add a blank data mark to the data row.
  • Step 312 Add a value data mark to the data row.
  • Step 314 Encrypt the message header and the data entity to form a binary file.
  • Step 316 Store the binary file as a data file.
  • Step 318 When reading the data object, decrypt the binary file to obtain the message header and the data entity.
  • Step 320 Load the message header and the data entity into the memory.
  • Step 322 When multiple data objects are stored, polling and monitoring the data version number of the message header and the data entity and the file version number of the data file in the memory.
  • Step 324 Determine whether the data object is updated according to the data version number and the file version number.
  • the data reading and writing method provided in this application splits the data file into a message header and data entities, and establishes an association relationship between the message header and the data entity through the mark of a private header, and marks each data line to To ensure the integrity of the data, the message header and data entity are encrypted with the private key to obtain and store the binary message header and data entity.
  • the binary file occupies a small space, and can be decrypted quickly, and can improve the data file Security.
  • the method of directly storing to the local file does not need to consider the installation of the database. Because the limitation of the database is removed, no matter what It is faster to read data files or write data files.
  • the data entity when the application is started, the data entity is loaded into the memory, which can increase the data file reading speed, so as to shorten the data file call time.
  • the message header and the data entity can be updated according to the updated data file to realize the persistent data file to the local file and realize the real update operation of the data file.
  • the underlying operation of the data object is placed in the base class of the application data operation layer, and a parameterless constructor is created in the base class to initialize the data object.
  • the underlying operation of the data file is placed in the base class of the application data operation layer.
  • a non-parameter constructor is created to realize the business of data operation initialization. In this way, when reading data When files are instantiated, each operation has a set of independent data operation transactions, so that multiple data entities are stored independently of each other without affecting each other.
  • it further includes: adding a synchronous safety lock to the external data operation interface of the data file.
  • a synchronization security lock is added to the external data operation interface to avoid data anomalies caused by multiple modification of the message header and data entity at the same time.
  • the method before the step of splitting the data object into a message header and a data entity, the method further includes: receiving the data object and determining the capacity of the data object; when the capacity is less than a preset threshold, executing the data object Steps to split the message header and data entities.
  • the threshold is preset Should be less than the capacity of the local file.
  • the preset threshold may be set according to the capacity of the local file, for example: 10MB, 20MB, 100MB, 10GB, etc.
  • the following shows a data file type in a data reading and writing method:
  • the enumeration class (JDataType) of Java basic type read and write operations is taken as an example for description: this class mainly includes "data type unique identifier (byted)" and “corresponding to Java Development Kit (Java Development Kit, JDK) "standard type (jdkType)" two attributes, at the same time provides a constructor with parameters, in order to fully meet business needs as much as possible, the realization of integer, long integer, (high-precision) floating point, enumeration Read and write operations of basic types of data such as type, Boolean type, character type, and date type.
  • Figure 4 shows the core data storage structure for reading and writing data files.
  • HEAD data description information
  • BODY stored value information
  • HEAD stores a header tag information, a key, the number of entity fields, and specific field information, only one copy is stored. And the data rows are stored in BODY, each data has a row mark information, and then the actual data information. If the field data is empty, it is directly stored in the empty data mark. If the field data has data information, it is stored in the value data Mark, followed by the actual data value.
  • Each data entity corresponds to a data file, and all data is stored in binary. In this way, while ensuring data integrity, the storage space is minimized, so that the data can be read efficiently and quickly (deserialization) and response Upper-level business operations.
  • DAO layer Data Access Objects layer, that is, data access object layer
  • base class create a no-parameter constructor to implement the business of data manipulation initialization.
  • the application starts and instantiates each DAO, it has its own set of data manipulation transactions.
  • loading data entities into the memory can increase the data reading speed, so as to shorten the response time of user operations and greatly improve the user operation experience.
  • the underlying implementation uses asynchronous multithreading. It has a task of polling and monitoring data writing. It can execute whether to generate a new data entity file or update a data entity file by judging whether the data version number and the file version number in the memory are the same.
  • the data reading method provided in this application can temporarily store the data in a local file, so that when the requirements are met, the deployment operation can be simplified, regardless of the installation of the database, and the processing of some data , When the amount of data in the memory reaches a certain level, the extra part can be stored in a local file according to a certain data format, and then deleted after the completion.
  • the upload and download of data files can also be implemented efficiently. Response effect.
  • the data reading method of the present application is highly efficient, and a set of callable data read and write interfaces is encapsulated, which is convenient for developers to develop and use.
  • the data read-write interface is applied to small and medium-sized projects that are quickly delivered.
  • the data reading method provided in this application implements basic types of read and write operations.
  • the operation of the data file object is realized by the custom read and write of the attributes.
  • the stored data is divided into the message header and the data body, which are all marked with an internal private header.
  • each data entity can be encrypted with a private key and in the form of binary data Stored locally to ensure the safety of data files.
  • the bottom layer is realized by multi-threading technology, and the data entities and message headers are cached to the memory at the same time, and the data version number and file version number in the memory are polled and monitored to compare the data version number and file version number in the memory to persist the data to the local file , Realize the real update operation of data.
  • the externally provided data operation interfaces are all equipped with synchronous safety locks to ensure that the same data entity will not be modified by multiple people at the same time, which may cause data abnormalities.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请提出了一种数据读写方法、计算机设备与计算机可读存储介质，其中，数据读写方法包括：将数据对象拆分为消息头与数据实体；在消息头中存放头部标记信息、数据实体的字段数量与字段信息；在数据实体中存放数据行，对数据行添加标记信息；将消息头与数据实体存储至本地文件。

Description

数据读写方法、计算机设备与计算机可读存储介质
本申请要求在2019年08月16日提交中国专利局、申请号为201910760034.3的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请涉及数据存储技术领域，例如涉及一种数据读写方法、一种计算机设备与一种计算机可读存储介质。
背景技术
相关技术中，对于某些离线项目，在将数据临时存储到本地文件时，通常直接存入数据库，而在读取时，需要在数据库中查找数据文件，并将文件调出数据库，而将数据存入或调出数据库将导致数据读写速度较慢。
发明内容
本申请至少解决相关技术中数据读写速度较慢的技术问题。本申请的第一方面实施例提出了一种读写快速的数据读写方法。
本申请的第二方面实施例提出了一种计算机设备。
本申请的第三方面实施例提出了一种计算机可读存储介质。
根据本申请的第一方面实施例，本申请提出了一种数据读写方法，包括：将数据对象拆分为消息头与数据实体；在消息头中存放头部标记信息、数据实体的字段数量与字段信息；在数据实体中存放数据行，对数据行添加标记信息；将消息头与数据实体存储至本地文件。
根据本申请的第二方面实施例，本申请提出了一种计算机设备，包括：处理器与存储器，存储器中存储有计算机程序，计算机程序在被处理器执行时，实现如上述技术方案中任一项所述的数据读写方法。
根据本申请的第三方面实施例，本申请提出了一种计算机存储介质，计算机存储介质上存储有计算机程序，计算机程序在被执行时，实现如上述技术方案中任一项所述的数据读写方法。
附图说明
图1示出本申请第一个实施例提供的数据读写方法的流程图；
图2示出本申请第二个实施例提供的数据读写方法的流程图；
图3示出本申请第三个实施例提供的数据读写方法的流程图；
图4示出本申请第三个实施例提供的数据读写方法中数据文件的存储结构示意图。
具体实施方式
下面结合附图和实施例对本申请进行描述。
下面参照图1至图4描述根据本申请一些实施例提供的数据读写方法、计算机设备与计算机可读存储介质。
图1示出本申请第一个实施例提供的数据读写方法的流程图。
如图1所示，本申请第一个实施例所提供的数据读写方法的流程包括如下步骤：
步骤102：将数据对象拆分为消息头与数据实体。
步骤104：在消息头中存放头部标记信息、数据实体的字段数量与字段信息。
步骤106：在数据实体中存放数据行，对数据行添加标记信息。
步骤108：将消息头与数据实体存储至本地文件。
本申请提出的数据读写方法，将数据文件拆分为消息头与数据实体，并在消息头与数据实体之间通过私有头的标记建立关联关系，再将消息头与数据实体直接存储在本地文件内，在读取文件时，可以直接查找消息头，再读取与消息头相对应的数据实体，而采用直接存储到本地文件的方式无需考虑数据库的安装，由于取消了数据库的限制，因此，无论是读取数据文件还是写入数据文件均具备较快的速度。
图2示出本申请第二个实施例提供的数据读写方法的流程图。
如图2所示，本申请第二个实施例所提供的数据读写方法的流程包括如下步骤：
步骤202：将数据对象拆分为消息头与数据实体。
步骤204：在消息头中存放头部标记信息、数据实体的字段数量与字段信息。
步骤206：在数据实体中存放数据行。
步骤208：判断数据行是否具有数据信息；在判断为否的情况下，执行步骤210，在判断结果为是的情况下，执行步骤212。
步骤210：为数据行添加空数据标记。
步骤212：为数据行添加有值数据标记。
步骤214：对消息头与数据实体进行加密，形成二进制文件。
步骤216：将二进制文件存储为一个数据文件。
本申请提出的数据读写方法，将数据文件拆分为消息头与数据实体，并在消息头与数据实体之间通过私有头的标记建立关联关系，再将消息头与数据实体直接存储在本地文件内，在读取文件时，可以直接查找消息头，再读取与消息头相对应的数据实体，而采用直接存储到本地文件的方式无需考虑数据库的安装，由于取消了数据库的限制，因此，无论是读取数据文件还是写入数据文件均具备较快的速度。
本实施例中，对每个数据行均进行标记，以保证数据的完整性。
本实施例中，以私钥对消息头与数据实体进行加密，以得到二进制的消息头与数据实体并进行存储，二进制的文件占用空间小，并且，能够快速解密，同时能够提升数据文件的安全性，并且，将消息头与数据实体存储为一个数据文件，能够保证在存储多个数据对象时，多个数据文件互不干涉。
图3示出本申请第三个实施例提供的数据读写方法的流程图。
如图3所示，本申请第三个实施例所提供的数据读写方法的流程包括如下步骤：
步骤302：将数据对象拆分为消息头与数据实体。
步骤304：在消息头中存放头部标记信息、数据实体的字段数量与字段信息。
步骤306：在数据实体中存放数据行。
步骤308：判断数据行是否具有数据信息；在判断结果为是情况下，执行步骤312，在判断结果为否的情况下，执行步骤310。
步骤310：为数据行添加空数据标记。
步骤312：为数据行添加有值数据标记。
步骤314：对消息头与数据实体进行加密，形成二进制文件。
步骤316：将二进制文件存储为一个数据文件。
步骤318：在读取数据对象时，对二进制文件进行解密，得到消息头与数据实体。
步骤320：将消息头与数据实体加载至内存。
步骤322：在存储有多个数据对象时，轮询监听所述内存中消息头与数据实体的数据版本号与数据文件的文件版本号。
步骤324：根据数据版本号与文件版本号，判断数据对象是否更新。
本申请提供的数据读写方法，将数据文件拆分为消息头与数据实体，并在消息头与数据实体之间通过私有头的标记建立关联关系，并对每个数据行均进行标记，以保证数据的完整性，以私钥对消息头与数据实体进行加密，以得到二进制的消息头与数据实体并进行存储，二进制的文件占用空间小，并且，能够快速解密，同时能够提升数据文件的安全性，在读取文件时，可以直接查找消息头，再根据关联关系直接读取数据实体，而采用直接存储到本地文件的方式无需考虑数据库的安装，由于取消了数据库的限制，因此，无论是读取数据文件还是写入数据文件均具备较快的速度。
本实施例中，在应用启动时，加载数据实体到内存中，可以提高数据文件读取速度，以便缩短数据文件的调用时间。
本实施例中，轮询监听内存中加载的消息头与数据实体的数据版本号，以及数据文件的文件版本号，对比数据版本号与文件版本号，若两者不同，则说明书数据文件有更新，此时，可根据更新后的数据文件更新消息头与数据实体，实现持久化数据文件到本地文件，实现数据文件的真实更新操作。
在本申请的一个实施例中，将数据对象的底层操作放置到应用数据操作层的基类中，并在基类中创建无参构造器，以对数据对象进行初始化。
在该实施例中，将数据文件的底层操作放到应用数据操作层的基类中，同时，在基类中，创建一个无参构造器，实现数据操作初始化的业务，这样，在读取数据文件时，实例化每个操作时，都有一套独立的数据操作事务，实现多个数据实体在存储上互相独立，互不影响。
在本申请的一个实施例中，还包括：在数据文件的对外数据操作接口加持同步安全锁。
在该实施例中，在对外数据操作接口加持同步安全锁，以避免消息头与数据实体同时被多方修改，而导致的数据异常。
在本申请的一个实施例中，在将数据对象拆分为消息头与数据实体的步骤之前，还包括：接收数据对象，判断数据对象的容量；在容量小于预设阈值时，执行将数据对象拆分为消息头与数据实体的步骤。
在该实施例中，由于数据对象是存储在本地文件中，因此，为了保证存储的可行性，数据对象的容量不能大于本地文件的容量，并且，为了能够同时存储多个数据对象，预设阈值应小于本地文件的容量。
在一实施例中，预设阈值可以根据本地文件的容量设置，例如：10MB、20MB、100MB、10GB等。
下方示出一种数据读写方法中一种数据文件的类型：
Figure PCTCN2020091814-appb-000001
在一实施例中，以Java基础类型读写操作的枚举类(JDataType)为例进行说明：该类主要有“数据类型唯一标识(byted)”和“对应Java开发工具包(Java Development Kit，JDK)中标准类型(jdkType)”两个属性，同时提供了一个带参构造器，为了尽可能全面的满足业务需求，实现了整型、长整型、(高精度)浮点型、枚举型、布尔型、字符型、日期类型等基础类型数据的读写操作。
对数据文件的读写，核心数据存储结构如图4所示。
分为数据描述信息(HEAD，即消息头)和存储数值信息(BODY，即数据 实体)。HEAD中存放一个头部标记信息、一个密钥、实体字段数量，以及具体的字段信息，仅存一份。而BODY中存放数据行，每条数据都有一个行标记信息，然后就是实际的数据信息，若字段数据为空，则直接存入空数据标记，若字段数据有数据信息，存入有值数据标记，后面紧跟实际数据值。每个数据实体对应一个数据文件，所有数据采用二进制存储，这样，在确保数据完整性的情况下，做到存储空间占用最小化，以便能够有效的快速的读取数据(反序列化)，响应上层业务操作。
为了确保多个数据实体在存储上互相独立，互不影响，我们将数据的底层操作放到应用数据操作层(DAO层，Data Access Objects层，即数据访问对象层)的基类中，同时，在基类中，创建一个无参构造器，实现数据操作初始化的业务。这样，在应用启动，实例化每个DAO时，都有一套自己的数据操作事务。应用启动时，加载数据实体到内存中，可以提高数据读取速度，以便缩短用户操作的响应时间，大大改善用户操作体验。
为了提高多个数据实体同时更新数据文件的效率，底层实现采用异步多线程处理。它里面有个轮询监听数据写的任务，通过判断内存中数据版本号和文件版本号是否相同，来执行是否生成新的数据实体文件或者更新数据实体文件。
本申请提供的数据读取方法，对于某些离线项目而言，可将数据临时存储到本地文件，这样在满足需求的情况下，可以简化部署操作，不用考虑数据库的安装，对于一些数据的处理，当内存数据量达到一定程度的时候，可将多出的部分按照某种数据格式存储到本地文件，完事后，再将其删除即可，对于数据文件的上传、下载，也能够实现高效地响应效果。
本实施例中，本申请的数据读取方法，具有高效性，并且封装出一套可供调用的数据读写接口，方便开发者开发使用。
例如，将所述数据读写接口应用到快速交付的中小型项目中。
在一实施例中，本申请提供的数据读取方法，实现基本类型的读写操作。
数据文件对象的操作统一采用属性的自定义读写实现，存储数据分为消息头和数据体，均用内部私有头标记标识，同时每个数据实体可使用私钥进行加密，并以二进制数据形式存储到本地，保证了数据文件的安全性。
为了满足多数据同时操作的场景，底层采用多线程技术实现，同时缓存数 据实体与消息头到内存，并轮询监听、对比内存内的数据版本号和文件版本号，来持久化数据到本地文件，实现数据的真实更新操作。
对外提供的数据操作接口都加同步安全锁，确保了同一数据实体不会同时被多人修改，而导致数据异常情况。

Claims (10)

  1. 一种数据读写方法，包括：
    将数据对象拆分为消息头与数据实体；
    在所述消息头中存放头部标记信息、所述数据实体的字段数量与字段信息；
    在所述数据实体中存放数据行，对所述数据行添加标记信息；
    将所述消息头与所述数据实体存储至本地文件。
  2. 根据权利要求1所述的数据读写方法，其中，所述对所述数据行添加标记信息包括：
    判断所述数据行是否具有数据信息；
    在所述数据行具有数据信息的情况下，为所述数据行添加有值数据标记；
    在所述数据行不具有数据信息的情况下，为所述数据行添加空数据标记。
  3. 根据权利要求1所述的数据读写方法，其中，所述将所述消息头与所述数据实体存储至本地文件包括：
    对所述消息头与所述数据实体进行加密，形成二进制文件；
    将所述二进制文件存储为一个数据文件。
  4. 根据权利要求3所述的数据读写方法，还包括：
    将所述数据对象的底层操作放置到应用数据操作层的基类中，并在所述基类中创建无参构造器，以对所述数据对象进行初始化。
  5. 根据权利要求4所述的数据读写方法，还包括：
    在读取所述数据对象的情况下，对所述二进制文件进行解密，得到所述消息头与所述数据实体；
    将所述消息头与所述数据实体加载至内存。
  6. 根据权利要求5所述的数据读写方法，还包括：
    在存储有多个所述数据对象时，轮询监听所述内存中所述消息头与所述数据实体的数据版本号与所述数据文件的文件版本号；
    根据所述数据版本号与所述文件版本号，判断所述数据对象是否更新。
  7. 根据权利要求3所述的数据读写方法，还包括：
    在所述数据文件的对外数据操作接口加持同步安全锁。
  8. 根据权利要求1至5中任一项所述的数据读写方法，在所述将数据对象拆分为消息头与数据实体之前，还包括：
    接收所述数据对象，判断所述数据对象的容量；
    在所述容量小于预设阈值的情况下，执行所述将数据对象拆分为消息头与数据实体的步骤。
  9. 一种计算机设备，包括：处理器与存储器，所述存储器中存储有计算机程序，所述计算机程序在被所述处理器执行时，实现如上述权利要求1至8中任一项所述的数据读写方法。
  10. 一种计算机存储介质，存储有计算机程序，所述计算机程序在被执行时，实现如上述权利要求1至8中任一项所述的数据读写方法。
PCT/CN2020/091814 2019-08-16 2020-05-22 数据读写方法、计算机设备与计算机可读存储介质 WO2021031640A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910760034.3A CN110597463A (zh) 2019-08-16 2019-08-16 数据读写方法、计算机设备与计算机可读存储介质
CN201910760034.3 2019-08-16

Publications (1)

Publication Number Publication Date
WO2021031640A1 true WO2021031640A1 (zh) 2021-02-25

Family

ID=68854557

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/091814 WO2021031640A1 (zh) 2019-08-16 2020-05-22 数据读写方法、计算机设备与计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN110597463A (zh)
WO (1) WO2021031640A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110597463A (zh) * 2019-08-16 2019-12-20 深圳中兴网信科技有限公司 数据读写方法、计算机设备与计算机可读存储介质
CN112344956B (zh) * 2020-11-05 2022-02-25 腾讯科技（深圳）有限公司 地图显示方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102609452A (zh) * 2012-01-11 2012-07-25 上海西本网络科技有限公司 数据的存储方法及装置
CN103177026A (zh) * 2011-12-23 2013-06-26 北京新媒传信科技有限公司 数据管理方法和数据管理系统
CN109284252A (zh) * 2018-09-11 2019-01-29 武汉虹信通信技术有限责任公司 一种通信设备中轻量级文件系统管理方法
CN110597463A (zh) * 2019-08-16 2019-12-20 深圳中兴网信科技有限公司 数据读写方法、计算机设备与计算机可读存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101710920B (zh) * 2009-11-23 2015-05-13 中兴通讯股份有限公司 移动通信终端消息管理方法和消息管理系统
CN103200213A (zh) * 2012-01-09 2013-07-10 北京四维图新科技股份有限公司 远程信息服务系统的数据传输方法及装置
CN102916948B (zh) * 2012-09-29 2015-05-06 深圳市易联盛世科技有限公司 数据安全处理方法、装置及终端
CN106527958B (zh) * 2015-09-09 2020-03-10 阿里巴巴集团控股有限公司 用于分布式存储系统的写入数据、获取数据的方法和设备
CN107704604B (zh) * 2017-10-16 2020-09-18 中汇信息技术（上海）有限公司 一种消息持久化方法、服务器和计算机可读存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103177026A (zh) * 2011-12-23 2013-06-26 北京新媒传信科技有限公司 数据管理方法和数据管理系统
CN102609452A (zh) * 2012-01-11 2012-07-25 上海西本网络科技有限公司 数据的存储方法及装置
CN109284252A (zh) * 2018-09-11 2019-01-29 武汉虹信通信技术有限责任公司 一种通信设备中轻量级文件系统管理方法
CN110597463A (zh) * 2019-08-16 2019-12-20 深圳中兴网信科技有限公司 数据读写方法、计算机设备与计算机可读存储介质

Also Published As

Publication number Publication date
CN110597463A (zh) 2019-12-20

Similar Documents

Publication Publication Date Title
EP2551854B1 (en) System for updating an associative memory
CN104516776B (zh) 数据处理管理方法及信息处理设备
JP5387757B2 (ja) 並列データ処理システム、並列データ処理方法及びプログラム
TWI412945B (zh) 擷取以及保存來自或存至關聯資料庫之各物件的方法及系統
CN104881334B (zh) 缓存数据的防掉电保护方法及系统
TWI479341B (zh) 資訊系統中轉換資料之高處理能力，可靠複寫
WO2021031640A1 (zh) 数据读写方法、计算机设备与计算机可读存储介质
CN107368362A (zh) 一种对于磁盘读写数据的多线程/多进程无锁处理方法及系统
US9785687B2 (en) System and method for transparent multi key-value weighted attributed connection using uni-tag connection pools
CN110287044A (zh) 无锁共享内存处理方法、装置、电子设备及可读存储介质
JP6612826B2 (ja) コンテナ管理装置、コンテナ管理方法、及びコンテナ管理プログラム
US20200065313A1 (en) Extensible content object metadata
US20110289514A1 (en) Sharing and synchronization of objects
US8572627B2 (en) Providing supplemental semantics to a transactional queue manager
US11797715B2 (en) Erasing a cryptographic hem in a memory of a device in response to a change of an operating mode of the device
WO2022095366A1 (zh) 基于Redis的数据读取方法、装置、设备及可读存储介质
CN110322390A (zh) 用于控制处理的方法和系统
CN109582496A (zh) 一致性快照组的创建方法、装置及计算机可读存储介质
TW201626218A (zh) 傳遞ａｐｉ的相依性的技術
US20220156128A1 (en) Computing device, computing equipment and programmable scheduling method
US11698795B2 (en) Unified way to track user configuration on a live system
US10055139B1 (en) Optimized layout in a two tier storage
CN109983452A (zh) 用于连续可用的网络文件系统(nfs)状态数据的系统和方法
TW200533136A (en) Key cache management through multiple localities
US20110191549A1 (en) Data Array Manipulation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20855535

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20855535

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20855535

Country of ref document: EP

Kind code of ref document: A1