WO2019028988A1 - Data processing method, electronic device and computer readable storage medium - Google Patents

Data processing method, electronic device and computer readable storage medium Download PDF

Info

Publication number
WO2019028988A1
WO2019028988A1 PCT/CN2017/104534 CN2017104534W WO2019028988A1 WO 2019028988 A1 WO2019028988 A1 WO 2019028988A1 CN 2017104534 W CN2017104534 W CN 2017104534W WO 2019028988 A1 WO2019028988 A1 WO 2019028988A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
type
score
preset
cache
Prior art date
Application number
PCT/CN2017/104534
Other languages
French (fr)
Chinese (zh)
Inventor
何刘兴
Original Assignee
上海壹账通金融科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海壹账通金融科技有限公司 filed Critical 上海壹账通金融科技有限公司
Publication of WO2019028988A1 publication Critical patent/WO2019028988A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers

Definitions

  • the present application relates to the field of data processing technologies, and in particular, to a data processing method, an electronic device, and a computer readable storage medium.
  • the traditional real data analysis system will cache some data (intermediate data or result data) in memory for later use, avoiding reading the disk, thereby increasing the speed, but the system crashes. Or power outage can result in loss of memory data.
  • the present application provides a data processing method, an electronic device, and a computer readable storage medium for saving system resources.
  • the present application provides a data processing method, an electronic device, and a computer readable storage medium for saving system resources.
  • the first aspect of the present application provides a data processing method, including:
  • the update is corresponding a score of the type of data, wherein each type of data corresponds to a score, and the initialized score of each type of data is a preset value;
  • the preset persistence condition satisfies ⁇
  • the persistent data amount of each type of data is determined based on the scoring interval in which the scores of the various types of data in the cache are located, wherein different scoring intervals correspond to different persistent data. the amount;
  • a second aspect of the present application provides an electronic device, including: a memory, a processor, and computer readable instructions stored on the memory and executable on the processor, wherein the processor executes the computer readable
  • the instruction ⁇ implements the following steps:
  • the preset persistence condition satisfies ⁇ , based on the scoring interval in which the scores of the various types of data in the cache are located, the persistent data amount of each type of data is determined, wherein different scoring intervals correspond to different persistent data. the amount;
  • the third aspect of the present application provides an electronic device, including:
  • a monitoring module configured to listen to an operation event for various types of data in the cache
  • an update module configured to: when the listening module listens to an operation event for any type of data in the cache, based on a type to which the currently operated data belongs, and a preset and currently monitored operation event a type-associated scoring algorithm, which updates a score of a corresponding type of data, wherein each type of data corresponds to a score, and the initialized score of each type of data is a preset value;
  • a determining module configured to determine, according to a scoring interval in which the scores of the various types of data in the cache are located, the persistent data amount of each type of data is determined, wherein different scoring intervals are corresponding to each other Different amounts of persistent data;
  • the persistence module is configured to persist the various types of data in the cache based on the amount of persistent data of the various types of data determined by the determining module.
  • a fourth aspect of the present application provides a computer readable storage medium, where the computer readable storage medium is stored Having computer readable instructions, the computer readable instructions being executed by at least one processor, the steps of:
  • the amount of persistent data of each type of data is determined based on the scoring interval of the scores of the various types of data, wherein different scoring intervals correspond to different persistent data amounts; 0026] Persistence of various types of data in the cache based on the determined amount of persistent data of various types of data.
  • the scores of the corresponding types of data are updated based on the monitored operation events for various types of data, and when the preset persistence conditions are satisfied, based on various types of data in the cache.
  • the scoring interval of the score of ⁇ determines the amount of persistent data of various types of data, and persists various types of data in the cache based on the determined amount of persistent data of various types of data. Since the different scoring intervals correspond to different amounts of persistent data, and the scores of the various types of data are obtained based on the monitored operational events of the corresponding categories of data, the data in the cache can be persisted by the solution of the present application. Resolving, the persistence of various types of data is related to operational events, thereby avoiding wasted system resources caused by persisting all cached data each time, saving system resources to a certain extent.
  • FIG. 1 is a schematic flow chart of an embodiment of a data processing program method provided by the present application.
  • FIG. 2 is a schematic structural diagram of an embodiment of an electronic device provided by the present application
  • FIG. 3 is a schematic structural diagram showing the structure of a function module of a data processing program provided by the present application.
  • a data processing method is described in the embodiment of the present application.
  • the data processing method in the embodiment of the present application includes:
  • Step 101 Listen to an operation event for each type of data in the cache
  • the buffer is a data exchange buffer (also referred to as a Cache in some embodiments), and the cache is a high-speed data exchange memory. Since the cache runs much faster than the memory, the role of the cache is to help the hardware run faster. Therefore, in order to pursue the practicality of data analysis, some data (such as intermediate data or result data) is usually stored in the cache. For later use, to improve data reading speed.
  • a data exchange buffer also referred to as a Cache in some embodiments
  • the cache is a high-speed data exchange memory. Since the cache runs much faster than the memory, the role of the cache is to help the hardware run faster. Therefore, in order to pursue the practicality of data analysis, some data (such as intermediate data or result data) is usually stored in the cache. For later use, to improve data reading speed.
  • the cache can contain multiple types of data, for example, in redis (redis is a key-val ue storage system) cache, the types of data can be divided into the following five types: String (string), List (list), Hash ( Hash), Set (unordered and non-repeating collections), and ZSet (ordered and non-repeating collections).
  • redis is a key-val ue storage system
  • the types of data can be divided into the following five types: String (string), List (list), Hash ( Hash), Set (unordered and non-repeating collections), and ZSet (ordered and non-repeating collections).
  • the classification of the data in the cache may be implemented by referring to the prior art, and details are not described herein again.
  • step 101 an operation event for various types of data in the cache may be monitored, so as to proceed to step 102 after listening to the above operation event.
  • Step 102 When an operation event for any type of data in the cache is monitored, based on a type to which the currently operated data belongs, and a pre-designed sub-algorithm associated with the type of the currently monitored operation event, updating The score of the corresponding type of data;
  • each type of data corresponds to a score
  • the initialized score of each type of data is a preset value.
  • the initialization score of each type of data can be set to 0 in advance.
  • an associated scoring algorithm may be set in advance for each operation event.
  • an operation event for any type of data in the cache is listened to, based on the type of data currently being operated on, And a pre-designed sub-algorithm associated with the type of the currently monitored operational event, updating the score of the corresponding type of data.
  • the score interval in which the score is located may be positively correlated with the amount of persistent data, that is, the higher the score interval in which the score of a certain type of data is, the persistence of the data.
  • the amount of data is also higher.
  • the requirement for the updated data to be persisted is considered to be small, and thus the score of the data can be subtracted, and when the insertion event and the query event are monitored, the The data for the corresponding category is more important, so you can increase the score for that type of data.
  • the step 102 may include: if the type of the currently monitored operation event is a query event, adding a score of the corresponding type of data to the preset first score based on the type of the currently operated data Updating the score of the corresponding type of data; if the type of the currently monitored operation event is an insertion event, adding a score of the corresponding type of data to the preset second score based on the type of the currently operated data Updating the score of the corresponding type of data; if the type of the currently monitored operation event is an update event, subtracting the score of the corresponding type of data from the preset third score based on the type of the currently operated data The score for the corresponding type of data.
  • the embodiment of the present application may also preset other scoring algorithms for different types of operation events.
  • the above operational events may include but are not limited to: query events, insert events, and update events, which are not limited herein.
  • Step 103 When the preset persistence condition is satisfied, determining a persistent data amount of each type of data based on a score interval in which the scores of the various types of data in the cache are in the cache;
  • different scoring intervals correspond to different persistent data amounts.
  • the amount of persistent data of each type of data is determined based on the scoring interval in which the scores of the various types of data in the cache are located.
  • the preset persistence condition may be: a distance interval from a previous execution of the persistence to a preset interval threshold.
  • Step 103 may include: determining a persistent data amount of each type of data based on a scoring interval in which the scores of the various types of data are performed, when the interval between the previous executions of the previous durations reaches a preset interval threshold. That is, step 103 may determine the amount of persistent data of each type of data for a periodicity interval based on the scores of various types of data.
  • the above persistence conditions may also include other conditions or set to other conditions, for example, in the above Based on the first application scenario, the above persistence conditions may further include: a preset control is triggered (for example, a button named "Start Persistence" is triggered).
  • Step 103 may include: when the preset control is triggered, determining the persistent data of each type of data based on the scoring interval of the scores of the various data entities
  • Step 104 Perform persistence on various types of data in the cache based on the determined amount of persistent data of each type of data;
  • each type of data in the cache is persisted. For example, when the amount of persistent data of the character string data determined in step 103 is 20 megabytes, the string data in the cache is persisted and the amount of persistent data is not more than 20 megabytes. It should be noted that when the amount of data of a certain type of data in the cache is smaller than the amount of persistent data determined in step 103, the data in the cache can be persisted.
  • the score of the data of the corresponding category may also be reset, so that the score of the data of the corresponding category is restored to the score of the initial group.
  • the method further includes: prioritizing the high priority data priority.
  • high priority may be set for part of data (for example, important data), so that this part of data can be preferentially persisted in the process of persistence to reduce the probability of losing important data.
  • persistence refers to a process of saving data in a cache to a storage device (for example, a disk) that can be permanently saved.
  • a storage device for example, a disk
  • the specific process of persistence can be implemented by referring to the prior art, and will not be described here.
  • the data processing method in the embodiment of the present application may further include: when the data arrives in the buffer, the preset effective time associated with the data, Persistence of data that achieves the above effective time. Specifically, by default, all data in the cache can be set to a fixed value in the cache, such as a small number, after a piece of data is stored in the cache.
  • persisting the data for example, removing the data from the cache and persisting it to a disk or database.
  • an operation interface may be provided in the embodiment of the present application, where the user can customize the effective time of a certain data in the cache, and whether the user is persistent in the effective time, if the user selects the effective time to proceed Persistence, when the data reaches the preset in the cache The data is effectively correlated with the data, and the data is persisted.
  • the data processing method in the embodiment of the present application may be implemented by a data processing device, and the data processing device may be integrated into an electronic device such as a mobile phone, a server, a personal computer, or the like, which is not limited herein.
  • the scores of the corresponding types of data are updated based on the monitored operation events for various types of data, and when the preset persistence conditions are satisfied, based on various types of data in the cache.
  • the scoring interval in which the score of the actual score is located determines the amount of persistent data of each type of data, and persists various types of data in the cache based on the determined amount of persistent data of various types of data. Since the different scoring intervals correspond to different amounts of persistent data, and the scores of the various types of data are obtained based on the monitored operational events of the corresponding categories of data, the data in the cache can be persisted by the solution of the present application. Resolving, the persistence of various types of data is related to operational events, thereby avoiding wasted system resources caused by persisting all cached data each time, saving system resources to a certain extent.
  • FIG. 2 is a schematic diagram showing the operating environment of the computer readable instructions (for example, the data processing program) related to the data processing method provided by the embodiment of the present application, The description only shows the parts related to the embodiments of the present application.
  • the computer readable instructions for example, the data processing program
  • the foregoing data processing program is installed and runs in an electronic device.
  • the electronic device may include, but is not limited to, one or more memories 21 (only one is shown), one or more processors 22 (only one is shown), and the above-described memory 21 and processor 22 are connected by a bus 23.
  • Figure 2 shows only the electronic device with components 21-23, but it should be understood that not all illustrated components may be implemented, and more components (e.g., displays, etc.) or fewer components may be implemented instead.
  • the memory 21 may be an internal storage unit of an electronic device, such as a hard disk or memory of the electronic device, in some embodiments.
  • the memory 21 may also be an external storage device of the electronic device in other embodiments, such as a plug-in hard disk equipped with an electronic device, a smart memory card (SMC), and a Secure Digital (SD) card. Flash card (Flash
  • the memory 21 may also include both an internal storage unit of the electronic device and an external storage device.
  • the memory 21 is used to store application software installed on an electronic device and various types of data, such as program codes of the above data processing program.
  • the memory 21 can also be used to temporarily store the output or The data that will be output.
  • the processor 22 may be a Central Processing Unit (CPU) in some embodiments.
  • CPU Central Processing Unit
  • a microprocessor or other data processing chip for running program code or processing data stored in the memory 21, for example, executing the above data processing program and the like.
  • FIG. 3 is a functional block diagram of a data processing program corresponding to the data processing method according to Embodiment 1 according to an embodiment of the present application.
  • the data processing program may be divided into one or more modules, and the one or more modules are stored in the memory 21 and are processed by one or more processors (the processor 22 in this embodiment). ) executed to complete the application.
  • the above data processing program can be divided into a listening module 31, an updating module 32, a determining module 33, and a persistence module 34.
  • a module as referred to in the present application refers to a series of computer readable instruction instruction segments capable of performing a specific function, which is more suitable than the program for describing the execution of the above data processing program in the above electronic device.
  • the following description will specifically describe the functions of the listening module 31, the updating module 32, the determining module 33, and the persistence module 34.
  • the monitoring module 31 is configured to listen to an operation event for each type of data in the cache
  • the update module 32 is configured to: when the listening module 31 monitors an operation event for any type of data in the cache, based on a type to which the currently operated data belongs, and a preset and currently monitored operation event a type-associated scoring algorithm, which updates a score of a corresponding type of data, wherein each type of data corresponds to a score, and the initialized score of each type of data is a preset value;
  • the determining module 33 is configured to determine, according to a scoring interval in which the scores of the various types of data in the cache are located, the persistent data amount of each type of data is determined, wherein different scoring intervals are used. Corresponding to different amounts of persistent data;
  • the persistence module 34 is configured to persist the various types of data in the cache based on the amount of persistent data of the various types of data determined by the determining module 33.
  • the above-mentioned scoring interval is positively correlated with the amount of persistent data.
  • the update module 32 is specifically configured to: if the type of the operation event currently monitored by the monitoring module 31 is a query event, add the score of the corresponding type of data to the preset first score based on the type of the currently operated data. a value to update the score of the corresponding type of data; if the type of the operation event currently being monitored by the listening module 31 is an insertion event, the score of the corresponding type of data is added to the preset based on the type of the currently operated data.
  • Second point a value to update the score of the corresponding type of data; if the type of the operation event currently monitored by the listening module 31 is an update event, the score of the corresponding type of data is subtracted from the preset based on the type of the currently operated data. The third score, the score of the corresponding type of data.
  • the persistence module 34 is further configured to: prioritize high priority data.
  • the determining module 33 is specifically configured to: when the distance between the previous execution of the persistent interval reaches a preset interval threshold, determine each of the score intervals based on the scores of the various types of data The amount of persistent data for class data.
  • the persistence module 34 is further configured to: when the data arrives in the cache, the preset valid data associated with the data, and the data that reaches the valid time is persisted.
  • the scores of the corresponding types of data are updated based on the monitored operation events for various types of data, and when the preset persistence conditions are satisfied, based on various types of data in the cache.
  • the scoring interval in which the score of the actual score is located determines the amount of persistent data of each type of data, and persists various types of data in the cache based on the determined amount of persistent data of various types of data. Since the different scoring intervals correspond to different amounts of persistent data, and the scores of the various types of data are obtained based on the monitored operational events of the corresponding categories of data, the data in the cache can be persisted by the solution of the present application. Resolving, the persistence of various types of data is related to operational events, thereby avoiding wasted system resources caused by persisting all cached data each time, saving system resources to a certain extent.
  • each functional unit and module in the above system can be referred to The corresponding processes in the foregoing method embodiments are not described herein.
  • the disclosed apparatus and method can be implemented in other manners.
  • the system embodiment described above is merely illustrative.
  • the division of the above module or unit is only a logical function division, and the actual implementation may have another division manner, for example, multiple units or components may be combined. Either can be integrated into another system, or some features can be ignored, or not executed.
  • the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in electrical, mechanical or other form.
  • the units described above as separate components may or may not be physically distributed, and the components displayed as the units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple networks. On the unit. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or in the form of software functional modules.
  • the integrated modules described above if implemented in the form of software functional modules and sold or used as separate products, may be stored in a computer readable storage medium.
  • the medium includes a plurality of instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the above steps of the various embodiments of the embodiments of the present application.
  • the foregoing storage medium includes: a USB flash drive, a mobile hard disk, and a read only memory (R) OM, Read-Only Memory), Random Access Memory (RAM)
  • a variety of media that can store program code such as a disk or a disc.

Abstract

A data processing method, an electronic device and a computer readable storage medium, being able to effectively save system resources. The data processing method above comprises: monitoring an operation event regarding various data in a cache (101); when it is detected that an operation event regarding any type of data in the cache, updating the score of the data of a corresponding type according to the type to which the current operated data belongs, and a pre-set scoring algorithm associated with the type of the currently detected operation event (102); when a pre-set persistence condition is satisfied, determining persistent data volume of various types of data according to scoring intervals to which the real-time scores of various types of data in the cache belongs (103); and performing persistence of the various types of data in the cache according to the determined persistent data volume of the various types of data (104).

Description

[0001] 本申请要求于 2017年 08月 10日提交中国专利局、 申请号为 201710680232.X、 发 明名称为"数据处理方法、 电子设备及计算机可读存储介质"的中国专利申请的优 先权, 其全部内容通过引用结合在本申请中。  [0001] This application claims the priority of the Chinese Patent Application, filed on Aug. 10, 2017, the application Serial No. 201710680232.X, entitled "Data Processing Method, Electronic Device, and Computer Readable Storage Media", The entire contents of this application are incorporated herein by reference.
技术领域  Technical field
[0002] 本申请涉及数据处理技术领域, 具体涉及一种数据处理方法、 电子设备及计算 机可读存储介质。  The present application relates to the field of data processing technologies, and in particular, to a data processing method, an electronic device, and a computer readable storage medium.
背景技术  Background technique
[0003] 为了追求数据分析的实吋性, 传统实吋数据分析系统会将一些数据 (中间数据 或结果数据) 缓存在内存中以备后续使用, 避免读取磁盘, 进而提升速度, 但 系统崩溃或断电会导致内存数据丢失。  [0003] In order to pursue the practicality of data analysis, the traditional real data analysis system will cache some data (intermediate data or result data) in memory for later use, avoiding reading the disk, thereby increasing the speed, but the system crashes. Or power outage can result in loss of memory data.
[0004] 虽然目前存在周期性将缓存数据进行持久化的方案, 但是, 持久化的过程需要 占用一定的系统资源, 过度的持久化会导致不必要的系统资源浪费。 [0004] Although there are currently schemes for periodically persisting cached data, the process of persistence requires a certain amount of system resources, and excessive persistence may result in unnecessary waste of system resources.
技术问题  technical problem
[0005] 本申请提供一种数据处理方法、 电子设备及计算机可读存储介质, 用以节省系 统资源。  The present application provides a data processing method, an electronic device, and a computer readable storage medium for saving system resources.
问题的解决方案  Problem solution
技术解决方案  Technical solution
[0006] 本申请提供一种数据处理方法、 电子设备及计算机可读存储介质, 用以节省系 统资源。  The present application provides a data processing method, an electronic device, and a computer readable storage medium for saving system resources.
[0007] 本申请第一方面提供一种数据处理方法, 其特征在于, 包括:  The first aspect of the present application provides a data processing method, including:
[0008] 监听针对缓存中的各类数据的操作事件; [0008] listening for operational events for various types of data in the cache;
[0009] 当监听到针对缓存中的任一类数据的操作事件吋, 基于当前被操作的数据所属 的类型, 以及预设的与当前监听到的操作事件的类型关联的计分算法, 更新相 应类型的数据的分数, 其中, 每类数据对应一分数, 且各类数据的初始化的分 数为预设值; [0010] 当预设的持久化条件满足吋, 基于缓存中的各类数据实吋的分数所在的评分区 间, 确定各类数据的持久化数据量, 其中, 不同评分区间对应不同的持久化数 据量; [0009] When an operation event for any type of data in the cache is monitored, based on the type of data currently being operated, and a preset scoring algorithm associated with the type of the currently monitored operation event, the update is corresponding a score of the type of data, wherein each type of data corresponds to a score, and the initialized score of each type of data is a preset value; [0010] When the preset persistence condition satisfies 吋, the persistent data amount of each type of data is determined based on the scoring interval in which the scores of the various types of data in the cache are located, wherein different scoring intervals correspond to different persistent data. the amount;
[0011] 基于确定的各类数据的持久化数据量对缓存中的各类数据进行持久化。  [0011] persisting various types of data in the cache based on the determined amount of persistent data of various types of data.
[0012] 本申请第二方面提供一种电子设备, 上述电子设备包括存储器、 处理器及存储 在上述存储器上并可在上述处理器上运行的计算机可读指令, 上述处理器执行 上述计算机可读指令吋实现如下步骤:  [0012] A second aspect of the present application provides an electronic device, including: a memory, a processor, and computer readable instructions stored on the memory and executable on the processor, wherein the processor executes the computer readable The instruction 吋 implements the following steps:
[0013] 监听针对缓存中的各类数据的操作事件;  [0013] listening for operational events for various types of data in the cache;
[0014] 当监听到针对缓存中的任一类数据的操作事件吋, 基于当前被操作的数据所属 的类型, 以及预设的与当前监听到的操作事件的类型关联的计分算法, 更新相 应类型的数据的分数, 其中, 每类数据对应一分数, 且各类数据的初始化的分 数为预设值;  [0014] when an operation event for any type of data in the cache is monitored, based on the type to which the currently operated data belongs, and a preset scoring algorithm associated with the type of the currently monitored operation event, updating the corresponding a score of the type of data, wherein each type of data corresponds to a score, and the initialized score of each type of data is a preset value;
[0015] 当预设的持久化条件满足吋, 基于缓存中的各类数据实吋的分数所在的评分区 间, 确定各类数据的持久化数据量, 其中, 不同评分区间对应不同的持久化数 据量;  [0015] When the preset persistence condition satisfies 吋, based on the scoring interval in which the scores of the various types of data in the cache are located, the persistent data amount of each type of data is determined, wherein different scoring intervals correspond to different persistent data. the amount;
[0016] 基于确定的各类数据的持久化数据量对缓存中的各类数据进行持久化。  [0016] Persistence of various types of data in the cache based on the determined amount of persistent data of various types of data.
[0017] 本申请第三方面提供一种电子设备, 包括:  [0017] The third aspect of the present application provides an electronic device, including:
[0018] 监听模块, 用于监听针对缓存中的各类数据的操作事件;  [0018] a monitoring module, configured to listen to an operation event for various types of data in the cache;
[0019] 更新模块, 用于当所述监听模块监听到针对缓存中的任一类数据的操作事件吋 , 基于当前被操作的数据所属的类型, 以及预设的与当前监听到的操作事件的 类型关联的计分算法, 更新相应类型的数据的分数, 其中, 每类数据对应一分 数, 且各类数据的初始化的分数为预设值;  [0019] an update module, configured to: when the listening module listens to an operation event for any type of data in the cache, based on a type to which the currently operated data belongs, and a preset and currently monitored operation event a type-associated scoring algorithm, which updates a score of a corresponding type of data, wherein each type of data corresponds to a score, and the initialized score of each type of data is a preset value;
[0020] 确定模块, 用于当预设的持久化条件满足吋, 基于缓存中的各类数据实吋的分 数所在的评分区间, 确定各类数据的持久化数据量, 其中, 不同评分区间对应 不同的持久化数据量;  [0020] a determining module, configured to determine, according to a scoring interval in which the scores of the various types of data in the cache are located, the persistent data amount of each type of data is determined, wherein different scoring intervals are corresponding to each other Different amounts of persistent data;
[0021] 持久化模块, 用于基于所述确定模块确定的各类数据的持久化数据量对缓存中 的各类数据进行持久化。  [0021] The persistence module is configured to persist the various types of data in the cache based on the amount of persistent data of the various types of data determined by the determining module.
[0022] 本申请第四方面提供一种计算机可读存储介质, 上述计算机可读存储介质存储 有计算机可读指令, 上述计算机可读指令被至少一个处理器执行吋, 实现如下 步骤: [0022] A fourth aspect of the present application provides a computer readable storage medium, where the computer readable storage medium is stored Having computer readable instructions, the computer readable instructions being executed by at least one processor, the steps of:
[0023] 监听对缓存中的各类数据的操作事件;  [0023] listening for operational events on various types of data in the cache;
[0024] 当监听到对缓存中的任一类数据的操作事件吋, 基于当前被操作的数据所属的 类型, 以及与当前监听到的操作事件的类型关联的预设计分算法, 更新相应类 型的数据的分数, 其中, 每类数据对应一分数, 且各类数据的初始化的分数为 预设值;  [0024] when an operation event for any type of data in the cache is monitored, based on the type to which the currently operated data belongs, and a pre-designed sub-algorithm associated with the type of the currently-listed operational event, updating the corresponding type a score of the data, wherein each type of data corresponds to a score, and the initialized score of each type of data is a preset value;
[0025] 当预设的持久化条件满足吋, 基于各类数据实吋的分数所在的评分区间, 确定 各类数据的持久化数据量, 其中, 不同评分区间对应不同的持久化数据量; [0026] 基于确定的各类数据的持久化数据量对缓存中的各类数据进行持久化。  [0025] When the preset persistence condition satisfies 吋, the amount of persistent data of each type of data is determined based on the scoring interval of the scores of the various types of data, wherein different scoring intervals correspond to different persistent data amounts; 0026] Persistence of various types of data in the cache based on the determined amount of persistent data of various types of data.
发明的有益效果  Advantageous effects of the invention
有益效果  Beneficial effect
[0027] 由上可见, 本申请方案中基于监听到的针对各类数据的操作事件对相应类型的 数据的分数进行更新, 当预设的持久化条件满足吋, 基于缓存中的各类数据实 吋的分数所在的评分区间, 确定各类数据的持久化数据量, 并基于确定的各类 数据的持久化数据量对缓存中的各类数据进行持久化。 由于不同评分区间对应 不同的持久化数据量, 且各类数据的分数是基于监听到的对相应类别的数据的 操作事件更新得到, 因此, 通过本申请方案, 可以在对缓存中的数据进行持久 化吋, 将各类数据的持久化量与操作事件相关, 从而避免每次都对全部缓存数 据进行持久化而导致的系统资源浪费, 在一定程度上节省了系统资源。  [0027] It can be seen that, in the solution of the present application, the scores of the corresponding types of data are updated based on the monitored operation events for various types of data, and when the preset persistence conditions are satisfied, based on various types of data in the cache. The scoring interval of the score of 吋 determines the amount of persistent data of various types of data, and persists various types of data in the cache based on the determined amount of persistent data of various types of data. Since the different scoring intervals correspond to different amounts of persistent data, and the scores of the various types of data are obtained based on the monitored operational events of the corresponding categories of data, the data in the cache can be persisted by the solution of the present application. Resolving, the persistence of various types of data is related to operational events, thereby avoiding wasted system resources caused by persisting all cached data each time, saving system resources to a certain extent.
对附图的简要说明  Brief description of the drawing
附图说明  DRAWINGS
[0028] 为了更清楚地说明本申请实施例或现有技术中的技术方案, 下面将对实施例或 现有技术描述中所需要使用的附图作简单地介绍, 显而易见地, 下面描述中的 附图仅仅是本申请的一些实施例, 对于本领域普通技术人员来讲, 在不付出创 造性劳动性的前提下, 还可以根据这些附图获得其他的附图。  [0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings to be used in the embodiments or the prior art description will be briefly described below, and obviously, in the following description The drawings are only some of the embodiments of the present application, and other drawings may be obtained from those skilled in the art without departing from the drawings.
[0029] 图 1为本申请提供的数据处理程序方法一个实施例流程示意图;  1 is a schematic flow chart of an embodiment of a data processing program method provided by the present application;
[0030] 图 2为本申请提供的电子设备一个实施例结构示意图; [0031] 图 3为本申请提供的数据处理程序的功能模块结构示意图结构示意图。 2 is a schematic structural diagram of an embodiment of an electronic device provided by the present application; [0031] FIG. 3 is a schematic structural diagram showing the structure of a function module of a data processing program provided by the present application.
[0032] 本申请的实施方式 [0032] Embodiments of the present application
[0033] 为使得本申请的发明目的、 特征、 优点能够更加的明显和易懂, 下面将结合本 申请实施例中的附图, 对本申请实施例中的技术方案进行清楚、 完整地描述, 显然, 所描述的实施例仅仅是本申请一部分实施例, 而非全部实施例。 基于本 申请中的实施例, 本领域普通技术人员在没有做出创造性劳动前提下所获得的 所有其他实施例, 都属于本申请保护的范围。  [0033] In order to make the object, the features and the advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present application. The described embodiments are only a part of the embodiments of the present application, and not all of them. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.
[0034] 实施例一  [0034] Embodiment 1
[0035] 本申请实施例对一种数据处理方法进行描述, 请参阅图 1, 本申请实施例中的 数据处理方法包括:  [0035] A data processing method is described in the embodiment of the present application. Referring to FIG. 1, the data processing method in the embodiment of the present application includes:
[0036] 步骤 101、 监听针对缓存中的各类数据的操作事件; [0036] Step 101: Listen to an operation event for each type of data in the cache;
[0037] 本申请实施例中, 缓存为数据交换的缓冲区 (在某些实施例也被称为 Cache) , 缓存是高速数据交换的存储器。 由于缓存的运行速度比内存快得多, 故缓存 的作用就是帮助硬件更快地运行, 因此, 为了追求数据分析的实吋性, 通常会 将一些数据 (例如中间数据或结果数据) 存入缓存中以备后续使用, 以提升数 据读取速度。 缓存中可以包含多种类型的数据, 例如在 redis (redis是一个 key-val ue存储系统) 缓存中, 数据的类型可分为如下 5种: String (字符串) 、 List (列 表) 、 Hash (哈希) 、 Set (无序且不重复集合) 和 ZSet (有序且不重复集合) 。 具体地, 关于缓存中数据的分类可以参照已有技术实现, 此处不再赘述。  [0037] In the embodiment of the present application, the buffer is a data exchange buffer (also referred to as a Cache in some embodiments), and the cache is a high-speed data exchange memory. Since the cache runs much faster than the memory, the role of the cache is to help the hardware run faster. Therefore, in order to pursue the practicality of data analysis, some data (such as intermediate data or result data) is usually stored in the cache. For later use, to improve data reading speed. The cache can contain multiple types of data, for example, in redis (redis is a key-val ue storage system) cache, the types of data can be divided into the following five types: String (string), List (list), Hash ( Hash), Set (unordered and non-repeating collections), and ZSet (ordered and non-repeating collections). Specifically, the classification of the data in the cache may be implemented by referring to the prior art, and details are not described herein again.
[0038] 在步骤 101中, 可以实吋监听针对缓存中的各类数据的操作事件, 以便在监听 到上述操作事件吋进入步骤 102。  [0038] In step 101, an operation event for various types of data in the cache may be monitored, so as to proceed to step 102 after listening to the above operation event.
[0039] 步骤 102、 当监听到针对缓存中的任一类数据的操作事件吋, 基于当前被操作 的数据所属的类型, 以及与当前监听到的操作事件的类型关联的预设计分算法 , 更新相应类型的数据的分数;  [0039] Step 102: When an operation event for any type of data in the cache is monitored, based on a type to which the currently operated data belongs, and a pre-designed sub-algorithm associated with the type of the currently monitored operation event, updating The score of the corresponding type of data;
[0040] 本申请实施例中, 每类数据对应一分数, 且各类数据的初始化的分数为预设值 。 例如, 可以预先设置各类数据的初始化的分数为 0。  [0040] In the embodiment of the present application, each type of data corresponds to a score, and the initialized score of each type of data is a preset value. For example, the initialization score of each type of data can be set to 0 in advance.
[0041] 本申请实施例中, 可以预先针对每种操作事件设置关联的计分算法。 当监听到 针对缓存中的任一类数据的操作事件吋, 基于当前被操作的数据所属的类型, 以及与当前监听到的操作事件的类型关联的预设计分算法, 更新相应类型的数 据的分数。 [0041] In the embodiment of the present application, an associated scoring algorithm may be set in advance for each operation event. When an operation event for any type of data in the cache is listened to, based on the type of data currently being operated on, And a pre-designed sub-algorithm associated with the type of the currently monitored operational event, updating the score of the corresponding type of data.
[0042] 在一种应用场景中, 可以将分数所在的评分区间与持久化数据量设为正相关的 关系, 即, 使得某类数据的分数所在的评分区间越高吋, 该类数据的持久化数 据量也越高。 在此应用场景下, 当监听到更新事件吋, 可认为被更新的数据持 久化的需求较小, 此吋可以减去该类数据的分数, 而当监听到插入事件和査询 事件吋, 认为相应类别的数据较为重要, 此吋可以增加该类数据的分数。 因此 , 步骤 102可以包括: 若当前监听到的操作事件的类型为査询事件, 则基于当前 被操作的数据所属的类型, 将相应类型的数据的分数加上预设的第一分值, 以 更新相应类型的数据的分数; 若当前监听到的操作事件的类型为插入事件, 则 基于当前被操作的数据所属的类型, 将相应类型的数据的分数加上预设的第二 分值, 以更新相应类型的数据的分数; 若当前监听到的操作事件的类型为更新 事件, 则基于当前被操作的数据所属的类型, 将相应类型的数据的分数减去预 设的第三分值, 以相应类型的数据的分数。  [0042] In an application scenario, the score interval in which the score is located may be positively correlated with the amount of persistent data, that is, the higher the score interval in which the score of a certain type of data is, the persistence of the data. The amount of data is also higher. In this application scenario, when an update event is detected, the requirement for the updated data to be persisted is considered to be small, and thus the score of the data can be subtracted, and when the insertion event and the query event are monitored, the The data for the corresponding category is more important, so you can increase the score for that type of data. Therefore, the step 102 may include: if the type of the currently monitored operation event is a query event, adding a score of the corresponding type of data to the preset first score based on the type of the currently operated data Updating the score of the corresponding type of data; if the type of the currently monitored operation event is an insertion event, adding a score of the corresponding type of data to the preset second score based on the type of the currently operated data Updating the score of the corresponding type of data; if the type of the currently monitored operation event is an update event, subtracting the score of the corresponding type of data from the preset third score based on the type of the currently operated data The score for the corresponding type of data.
[0043] 当然, 本申请实施例也可以针对不同类型的操作事件预设其它的计分算法。 上 述操作事件可包括但不限于: 査询事件、 插入事件和更新事件, 此处不做限定  [0043] Of course, the embodiment of the present application may also preset other scoring algorithms for different types of operation events. The above operational events may include but are not limited to: query events, insert events, and update events, which are not limited herein.
[0044] 步骤 103、 当预设的持久化条件满足吋, 基于缓存中的各类数据实吋的分数所 在的评分区间, 确定各类数据的持久化数据量; [0044] Step 103: When the preset persistence condition is satisfied, determining a persistent data amount of each type of data based on a score interval in which the scores of the various types of data in the cache are in the cache;
[0045] 由前述可知, 本申请实施例中, 不同评分区间对应不同的持久化数据量。 当预 设的持久化条件满足吋, 基于缓存中的各类数据实吋的分数所在的评分区间, 确定各类数据的持久化数据量。  [0045] As can be seen from the foregoing, in the embodiment of the present application, different scoring intervals correspond to different persistent data amounts. When the pre-established persistence condition is satisfied, the amount of persistent data of each type of data is determined based on the scoring interval in which the scores of the various types of data in the cache are located.
[0046] 在一种应用场景中, 预设的持久化条件可以为: 距离前一次执行所述持久化的 吋间间隔达到预设的间隔阈值。 则步骤 103可以包括:当距离前一次执行上述持 久化的吋间间隔达到预设的间隔阈值吋, 基于各类数据实吋的分数所在的评分 区间, 确定各类数据的持久化数据量。 也即, 步骤 103可以为周期性基于各类数 据实吋的分数所在的评分区间, 确定各类数据的持久化数据量。  [0046] In an application scenario, the preset persistence condition may be: a distance interval from a previous execution of the persistence to a preset interval threshold. Step 103 may include: determining a persistent data amount of each type of data based on a scoring interval in which the scores of the various types of data are performed, when the interval between the previous executions of the previous durations reaches a preset interval threshold. That is, step 103 may determine the amount of persistent data of each type of data for a periodicity interval based on the scores of various types of data.
[0047] 当然, 上述持久化条件还可以包括其它条件或设置为其它条件, 比如, 在上述 第一种应用场景的基础上, 上述持久化条件还可以包括: 预设的控件被触发 ( 例如名为"启动持久化"的按键被触发) 。 则步骤 103可以包括:当预设的控件被触 发吋, 基于各类数据实吋的分数所在的评分区间, 确定各类数据的持久化数据 [0047] Of course, the above persistence conditions may also include other conditions or set to other conditions, for example, in the above Based on the first application scenario, the above persistence conditions may further include: a preset control is triggered (for example, a button named "Start Persistence" is triggered). Step 103 may include: when the preset control is triggered, determining the persistent data of each type of data based on the scoring interval of the scores of the various data entities
[0048] 步骤 104、 基于确定的各类数据的持久化数据量对缓存中的各类数据进行持久 化; [0048] Step 104: Perform persistence on various types of data in the cache based on the determined amount of persistent data of each type of data;
[0049] 本申请实施例中基于步骤 103确定的各类数据的持久化数据量, 对缓存中的各 类数据进行持久化。 例如, 当步骤 103确定的字符串数据的持久化数据量为 20兆 吋, 对缓存中的字符串数据进行持久化且使得持久化的数据量不超过 20兆。 需 要说明的是, 当缓存中的某类数据的数据量小于步骤 103所确定的持久化数据量 吋, 可以将缓存中的该类数据都进行持久化。  [0049] In the embodiment of the present application, based on the persistent data amount of each type of data determined in step 103, each type of data in the cache is persisted. For example, when the amount of persistent data of the character string data determined in step 103 is 20 megabytes, the string data in the cache is persisted and the amount of persistent data is not more than 20 megabytes. It should be noted that when the amount of data of a certain type of data in the cache is smaller than the amount of persistent data determined in step 103, the data in the cache can be persisted.
[0050] 可选的, 在对相应类别的数据进行持久化之后, 还可以重置相应类别的数据的 分数, 以使得相应类别的数据的分数还原为初始化吋的分数。  [0050] Optionally, after the data of the corresponding category is persisted, the score of the data of the corresponding category may also be reset, so that the score of the data of the corresponding category is restored to the score of the initial group.
[0051] 可选的, 在步骤 104对各类数据进行持久化的过程中还包括: 对高优先级的数 据优先进行持久化。 本申请实施例中, 可以针对部分数据 (例如重要的数据) 设置高优先级, 以便在持久化的过程中可以优先对这部分数据进行持久化, 以 减少重要数据丢失的概率。  [0051] Optionally, in the step of persisting the various types of data in step 104, the method further includes: prioritizing the high priority data priority. In the embodiment of the present application, high priority may be set for part of data (for example, important data), so that this part of data can be preferentially persisted in the process of persistence to reduce the probability of losing important data.
[0052] 需要说明的是, 上述持久化是指将缓存中的数据保存到可永久保存的存储设备 中 (例如磁盘) 的过程。 持久化的具体过程可以参照已有技术实现, 此处不再 赘述。  [0052] It should be noted that the foregoing persistence refers to a process of saving data in a cache to a storage device (for example, a disk) that can be permanently saved. The specific process of persistence can be implemented by referring to the prior art, and will not be described here.
[0053] 可选的, 为了进一步减少数据丢失的概率, 本申请实施例中的数据处理方法还 可以包括: 当数据在缓存中的吋间到达预设的与该数据关联的有效吋间吋, 对 达到上述有效吋间的数据进行持久化。 具体的, 可以默认将所有数据在缓存中 的有效吋间设为一个固定值, 比如 1个小吋, 当某条数据存入缓存中一个小吋后  [0053] Optionally, in order to further reduce the probability of data loss, the data processing method in the embodiment of the present application may further include: when the data arrives in the buffer, the preset effective time associated with the data, Persistence of data that achieves the above effective time. Specifically, by default, all data in the cache can be set to a fixed value in the cache, such as a small number, after a piece of data is stored in the cache.
, 对该数据进行持久化 (例如, 将该数据从缓存中被清除并持久化进磁盘或数 据库) 。 或者, 本申请实施例中还可以提供一个操作界面, 用户可以在上面自 定义某条数据在缓存中的有效吋间, 以及有效吋间到了是否进行持久化, 若用 户选择了有效吋间到了进行持久化, 则当该数据在缓存中的吋间到达预设的与 该数据关联的有效吋间吋, 对该数据进行持久化。 , persisting the data (for example, removing the data from the cache and persisting it to a disk or database). Alternatively, an operation interface may be provided in the embodiment of the present application, where the user can customize the effective time of a certain data in the cache, and whether the user is persistent in the effective time, if the user selects the effective time to proceed Persistence, when the data reaches the preset in the cache The data is effectively correlated with the data, and the data is persisted.
[0054] 需要说明的是, 本申请实施例中的数据处理方法可以由数据处理装置实现, 该 数据处理装置具体可以集成在诸如手机、 服务器、 个人计算机等电子设备中, 此处不作限定。  [0054] It is to be noted that the data processing method in the embodiment of the present application may be implemented by a data processing device, and the data processing device may be integrated into an electronic device such as a mobile phone, a server, a personal computer, or the like, which is not limited herein.
[0055] 由上可见, 本申请实施例中基于监听到的针对各类数据的操作事件对相应类型 的数据的分数进行更新, 当预设的持久化条件满足吋, 基于缓存中的各类数据 实吋的分数所在的评分区间, 确定各类数据的持久化数据量, 并基于确定的各 类数据的持久化数据量对缓存中的各类数据进行持久化。 由于不同评分区间对 应不同的持久化数据量, 且各类数据的分数是基于监听到的对相应类别的数据 的操作事件更新得到, 因此, 通过本申请方案, 可以在对缓存中的数据进行持 久化吋, 将各类数据的持久化量与操作事件相关, 从而避免每次都对全部缓存 数据进行持久化而导致的系统资源浪费, 在一定程度上节省了系统资源。  [0055] It can be seen that, in the embodiment of the present application, the scores of the corresponding types of data are updated based on the monitored operation events for various types of data, and when the preset persistence conditions are satisfied, based on various types of data in the cache. The scoring interval in which the score of the actual score is located determines the amount of persistent data of each type of data, and persists various types of data in the cache based on the determined amount of persistent data of various types of data. Since the different scoring intervals correspond to different amounts of persistent data, and the scores of the various types of data are obtained based on the monitored operational events of the corresponding categories of data, the data in the cache can be persisted by the solution of the present application. Resolving, the persistence of various types of data is related to operational events, thereby avoiding wasted system resources caused by persisting all cached data each time, saving system resources to a certain extent.
[0056] 实施例二  [0056] Embodiment 2
[0057] 对应于实施例一所述的数据处理方法, 图 2示出了本申请实施例提供的与上述 数据处理方法相关的计算机可读指令 (例如数据处理程序) 的运行环境示意图 , 为了便于说明, 仅示出了与本申请实施例相关的部分。  [0057] Corresponding to the data processing method according to the first embodiment, FIG. 2 is a schematic diagram showing the operating environment of the computer readable instructions (for example, the data processing program) related to the data processing method provided by the embodiment of the present application, The description only shows the parts related to the embodiments of the present application.
[0058] 在本申请实施例中, 上述数据处理程序安装并运行于电子设备中。 该电子设备 可包括但不仅限于一个或多个存储器 21 (图中仅示出一个) 、 一个或多个处理 器 22 (图中仅示出一个) , 上述存储器 21和处理器 22通过总线 23连接。 图 2仅示 出了具有组件 21-23的电子设备, 但是应理解的是, 并不要求实施所有示出的组 件, 可以替代的实施更多的组件 (例如显示器等) 或者更少的组件。  [0058] In the embodiment of the present application, the foregoing data processing program is installed and runs in an electronic device. The electronic device may include, but is not limited to, one or more memories 21 (only one is shown), one or more processors 22 (only one is shown), and the above-described memory 21 and processor 22 are connected by a bus 23. . Figure 2 shows only the electronic device with components 21-23, but it should be understood that not all illustrated components may be implemented, and more components (e.g., displays, etc.) or fewer components may be implemented instead.
[0059] 存储器 21在一些实施例中可以是电子设备的内部存储单元, 例如该电子设备的 硬盘或内存。 存储器 21在另一些实施例中也可以是电子设备的外部存储设备, 例如电子设备上配备的插接式硬盘, 智能存储卡 (Smart Media Card, SMC) , 安全数字 (Secure Digital, SD) 卡, 闪存卡 (Flash  [0059] The memory 21 may be an internal storage unit of an electronic device, such as a hard disk or memory of the electronic device, in some embodiments. The memory 21 may also be an external storage device of the electronic device in other embodiments, such as a plug-in hard disk equipped with an electronic device, a smart memory card (SMC), and a Secure Digital (SD) card. Flash card (Flash
Card) 等。 进一步地, 存储器 21还可以既包括电子设备的内部存储单元也包括外 部存储设备。 存储器 21用于存储安装于电子设备的应用软件及各类数据, 例如 上述数据处理程序的程序代码等。 存储器 21还可以用于暂吋地存储已经输出或 者将要输出的数据。 Card) and so on. Further, the memory 21 may also include both an internal storage unit of the electronic device and an external storage device. The memory 21 is used to store application software installed on an electronic device and various types of data, such as program codes of the above data processing program. The memory 21 can also be used to temporarily store the output or The data that will be output.
[0060] 处理器 22在一些实施例中可以是一中央处理器 (Central Processing Unit, CPU) [0060] The processor 22 may be a Central Processing Unit (CPU) in some embodiments.
, 微处理器或其他数据处理芯片, 用于运行存储器 21中存储的程序代码或处理 数据, 例如执行上述数据处理程序等。 , a microprocessor or other data processing chip for running program code or processing data stored in the memory 21, for example, executing the above data processing program and the like.
[0061] 进一步, 在图 2基础上, 请参阅图 3, 图 3为本申请实施例提供的对应于实施例 一所述的数据处理方法的数据处理程序的功能模块图。 在本申请实施例中, 上 述数据处理程序可以被分割成一个或多个模块, 上述一个或者多个模块被存储 于存储器 21中, 并由一个或多个处理器 (本实施例为处理器 22) 所执行, 以完 成本申请。 例如, 在图 3中, 上述数据处理程序可以被分割成监听模块 31、 更新 模块 32、 确定模块 33和持久化模块 34。 本申请所称的模块是指能够完成特定功 能的一系列计算机可读指令指令段, 比程序更适合于描述上述数据处理程序在 上述电子设备中的执行过程。 以下描述将具体介绍监听模块 31、 更新模块 32、 确定模块 33和持久化模块 34的功能。  [0061] Further, based on FIG. 2, please refer to FIG. 3. FIG. 3 is a functional block diagram of a data processing program corresponding to the data processing method according to Embodiment 1 according to an embodiment of the present application. In the embodiment of the present application, the data processing program may be divided into one or more modules, and the one or more modules are stored in the memory 21 and are processed by one or more processors (the processor 22 in this embodiment). ) executed to complete the application. For example, in Fig. 3, the above data processing program can be divided into a listening module 31, an updating module 32, a determining module 33, and a persistence module 34. A module as referred to in the present application refers to a series of computer readable instruction instruction segments capable of performing a specific function, which is more suitable than the program for describing the execution of the above data processing program in the above electronic device. The following description will specifically describe the functions of the listening module 31, the updating module 32, the determining module 33, and the persistence module 34.
[0062] 监听模块 31, 用于监听针对缓存中的各类数据的操作事件;  [0062] The monitoring module 31 is configured to listen to an operation event for each type of data in the cache;
[0063] 更新模块 32, 用于当监听模块 31监听到针对缓存中的任一类数据的操作事件吋 , 基于当前被操作的数据所属的类型, 以及预设的与当前监听到的操作事件的 类型关联的计分算法, 更新相应类型的数据的分数, 其中, 每类数据对应一分 数, 且各类数据的初始化的分数为预设值;  [0063] The update module 32 is configured to: when the listening module 31 monitors an operation event for any type of data in the cache, based on a type to which the currently operated data belongs, and a preset and currently monitored operation event a type-associated scoring algorithm, which updates a score of a corresponding type of data, wherein each type of data corresponds to a score, and the initialized score of each type of data is a preset value;
[0064] 确定模块 33, 用于当预设的持久化条件满足吋, 基于缓存中的各类数据实吋的 分数所在的评分区间, 确定各类数据的持久化数据量, 其中, 不同评分区间对 应不同的持久化数据量;  [0064] The determining module 33 is configured to determine, according to a scoring interval in which the scores of the various types of data in the cache are located, the persistent data amount of each type of data is determined, wherein different scoring intervals are used. Corresponding to different amounts of persistent data;
[0065] 持久化模块 34, 用于基于确定模块 33确定的各类数据的持久化数据量对缓存中 的各类数据进行持久化。  [0065] The persistence module 34 is configured to persist the various types of data in the cache based on the amount of persistent data of the various types of data determined by the determining module 33.
[0066] 可选的, 上述评分区间与持久化数据量成正相关。 更新模块 32具体用于: 若监 听模块 31当前监听到的操作事件的类型为査询事件, 则基于当前被操作的数据 所属的类型, 将相应类型的数据的分数加上预设的第一分值, 以更新相应类型 的数据的分数; 若监听模块 31当前监听到的操作事件的类型为插入事件, 则基 于当前被操作的数据所属的类型, 将相应类型的数据的分数加上预设的第二分 值, 以更新相应类型的数据的分数; 若监听模块 31当前监听到的操作事件的类 型为更新事件, 则基于当前被操作的数据所属的类型, 将相应类型的数据的分 数减去预设的第三分值, 以相应类型的数据的分数。 [0066] Optionally, the above-mentioned scoring interval is positively correlated with the amount of persistent data. The update module 32 is specifically configured to: if the type of the operation event currently monitored by the monitoring module 31 is a query event, add the score of the corresponding type of data to the preset first score based on the type of the currently operated data. a value to update the score of the corresponding type of data; if the type of the operation event currently being monitored by the listening module 31 is an insertion event, the score of the corresponding type of data is added to the preset based on the type of the currently operated data. Second point a value to update the score of the corresponding type of data; if the type of the operation event currently monitored by the listening module 31 is an update event, the score of the corresponding type of data is subtracted from the preset based on the type of the currently operated data. The third score, the score of the corresponding type of data.
[0067] 可选的, 持久化模块 34还用于: 对高优先级的数据优先进行持久化。 [0067] Optionally, the persistence module 34 is further configured to: prioritize high priority data.
[0068] 可选的, 确定模块 33具体用于: 当距离前一次执行所述持久化的吋间间隔达到 预设的间隔阈值吋, 基于各类数据实吋的分数所在的评分区间, 确定各类数据 的持久化数据量。 [0068] Optionally, the determining module 33 is specifically configured to: when the distance between the previous execution of the persistent interval reaches a preset interval threshold, determine each of the score intervals based on the scores of the various types of data The amount of persistent data for class data.
[0069] 可选的, 持久化模块 34还用于: 当数据在缓存中的吋间到达预设的与该数据关 联的有效吋间吋, 对达到所述有效吋间的数据进行持久化。  [0069] Optionally, the persistence module 34 is further configured to: when the data arrives in the cache, the preset valid data associated with the data, and the data that reaches the valid time is persisted.
[0070] 应理解, 本申请实施例中的电子设备可以用于实现上述方法实施例中的全部技 术方案, 在本申请实施例中没有详述和提及的部分, 可以参见上述方法实施例 的描述, 此处不再赘述。  [0070] It should be understood that the electronic device in the embodiment of the present application may be used to implement all the technical solutions in the foregoing method embodiments, and the parts that are not detailed and mentioned in the embodiments of the present application may be referred to the foregoing method embodiments. Description, no longer repeat here.
[0071] 由上可见, 本申请实施例中基于监听到的针对各类数据的操作事件对相应类型 的数据的分数进行更新, 当预设的持久化条件满足吋, 基于缓存中的各类数据 实吋的分数所在的评分区间, 确定各类数据的持久化数据量, 并基于确定的各 类数据的持久化数据量对缓存中的各类数据进行持久化。 由于不同评分区间对 应不同的持久化数据量, 且各类数据的分数是基于监听到的对相应类别的数据 的操作事件更新得到, 因此, 通过本申请方案, 可以在对缓存中的数据进行持 久化吋, 将各类数据的持久化量与操作事件相关, 从而避免每次都对全部缓存 数据进行持久化而导致的系统资源浪费, 在一定程度上节省了系统资源。  [0071] It can be seen that, in the embodiment of the present application, the scores of the corresponding types of data are updated based on the monitored operation events for various types of data, and when the preset persistence conditions are satisfied, based on various types of data in the cache. The scoring interval in which the score of the actual score is located determines the amount of persistent data of each type of data, and persists various types of data in the cache based on the determined amount of persistent data of various types of data. Since the different scoring intervals correspond to different amounts of persistent data, and the scores of the various types of data are obtained based on the monitored operational events of the corresponding categories of data, the data in the cache can be persisted by the solution of the present application. Resolving, the persistence of various types of data is related to operational events, thereby avoiding wasted system resources caused by persisting all cached data each time, saving system resources to a certain extent.
[0072] 为了描述的方便和简洁, 仅以上述各功能单元、 模块的划分进行举例说明, 实 际应用中, 可以根据需要而将上述功能分配由不同的功能单元、 模块完成, 即 将上述装置的内部结构划分成不同的功能单元或模块, 以完成以上描述的全部 或者部分功能。 实施例中的各功能单元、 模块可以集成在一个处理单元中, 也 可以是各个单元单独物理存在, 也可以两个或两个以上单元集成在一个单元中 , 上述集成的单元既可以采用硬件的形式实现, 也可以采用软件功能单元的形 式实现。 另外, 各功能单元、 模块的具体名称也只是为了便于相互区分, 并不 用于限制本申请的保护范围。 上述系统中单元、 模块的具体工作过程, 可以参 考前述方法实施例中的对应过程, 在此不再赘述。 [0072] For convenience and brevity of description, only the division of each functional unit and module described above is exemplified. In practical applications, the above-mentioned function assignment can be completed by different functional units and modules as needed, that is, the inside of the above device. The structure is divided into different functional units or modules to perform all or part of the functions described above. Each functional unit and module in the embodiment may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit, and the integrated unit may be implemented by hardware. Formal implementation can also be implemented in the form of software functional units. In addition, the specific names of the functional units and modules are only for the purpose of distinguishing from each other, and are not intended to limit the scope of protection of the present application. The specific working process of the unit and module in the above system can be referred to The corresponding processes in the foregoing method embodiments are not described herein.
[0073] 本领域普通技术人员可以意识到, 结合本文中所公幵的实施例描述的各示例的 单元及算法步骤, 能够以电子硬件、 或者计算机软件和电子硬件的结合来实现 。 这些功能究竟以硬件还是软件方式来执行, 取决于技术方案的特定应用和设 计约束条件。 专业技术人员可以对每个特定的应用来使用不同方法来实现所描 述的功能, 但是这种实现不应认为超出本申请的范围。  [0073] Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present application.
[0074] 在本申请所提供的实施例中, 应该理解到, 所揭露的装置和方法, 可以通过其 它的方式实现。 例如, 以上所描述的系统实施例仅仅是示意性的, 例如, 上述 模块或单元的划分, 仅仅为一种逻辑功能划分, 实际实现吋可以有另外的划分 方式, 例如多个单元或组件可以结合或者可以集成到另一个系统, 或一些特征 可以忽略, 或不执行。 另一点, 所显示或讨论的相互之间的耦合或直接耦合或 通讯连接可以是通过一些接口, 装置或单元的间接耦合或通讯连接, 可以是电 性, 机械或其它的形式。  [0074] In the embodiments provided by the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the system embodiment described above is merely illustrative. For example, the division of the above module or unit is only a logical function division, and the actual implementation may have another division manner, for example, multiple units or components may be combined. Either can be integrated into another system, or some features can be ignored, or not executed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in electrical, mechanical or other form.
[0075] 上述作为分离部件说明的单元可以是或者也可以不是物理上分幵的, 作为单元 显示的部件可以是或者也可以不是物理单元, 即可以位于一个地方, 或者也可 以分布到多个网络单元上。 可以根据实际的需要选择其中的部分或者全部单元 来实现本实施例方案的目的。  [0075] The units described above as separate components may or may not be physically distributed, and the components displayed as the units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple networks. On the unit. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
[0076] 另外, 在本申请各个实施例中的各功能模块可以集成在一个处理模块中, 也可 以是各个模块单独物理存在, 也可以两个或两个以上模块集成在一个模块中。 上述集成的模块既可以采用硬件的形式实现, 也可以采用软件功能模块的形式 实现。  In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.
[0077] 上述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用 吋, 可以存储在一个计算机可读取存储介质中。 基于这样的理解, 本申请实施 例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部 或部分可以以软件产品的形式体现出来, 该计算机软件产品存储在一个存储介 质中, 包括若干指令用以使得一台计算机设备 (可以是个人计算机, 服务器, 或者网络设备等) 或处理器 (processor) 执行本申请实施例各个实施例上述方法 的全部或部分步骤。 而前述的存储介质包括: U盘、 移动硬盘、 只读存储器 (R OM , Read-Only Memory) 、 随机存取存储器 (RAM, Random Access Memory[0077] The integrated modules described above, if implemented in the form of software functional modules and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may contribute to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage. The medium includes a plurality of instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the above steps of the various embodiments of the embodiments of the present application. The foregoing storage medium includes: a USB flash drive, a mobile hard disk, and a read only memory (R) OM, Read-Only Memory), Random Access Memory (RAM)
) 、 磁碟或者光盘等各种可以存储程序代码的介质。 ) A variety of media that can store program code, such as a disk or a disc.
以上所述实施例仅用以说明本申请的技术方案, 而非对其限制; 尽管参照前述 实施例对本申请进行了详细的说明, 本领域的普通技术人员应当理解: 其依然 可以对前述各实施例所记载的技术方案进行修改, 或者对其中部分技术特征进 行等同替换; 而这些修改或者替换, 并不使相应技术方案的本质脱离本申请各 实施例技术方案的精神和范围, 均应包含在本申请的保护范围之内。  The above-mentioned embodiments are only used to explain the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that The technical solutions described in the examples are modified, or some of the technical features are equivalently replaced; and the modifications or substitutions do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in Within the scope of protection of this application.

Claims

权利要求书 Claim
[权利要求 1] 一种数据处理方法, 其特征在于, 包括: [Claim 1] A data processing method, comprising:
监听针对缓存中的各类数据的操作事件;  Listen for operational events for various types of data in the cache;
当监听到针对缓存中的任一类数据的操作事件吋, 基于当前被操作的 数据所属的类型, 以及预设的与当前监听到的操作事件的类型关联的 计分算法, 更新相应类型的数据的分数, 其中, 每类数据对应一分数 When an operation event for any type of data in the cache is monitored, the corresponding type of data is updated based on the type of the currently operated data and the preset scoring algorithm associated with the type of the currently monitored operation event. a score, where each type of data corresponds to a score
, 且各类数据的初始化的分数为预设值; , and the initialization score of each type of data is a preset value;
当预设的持久化条件满足吋, 基于缓存中的各类数据实吋的分数所在 的评分区间, 确定各类数据的持久化数据量, 其中, 不同评分区间对 应不同的持久化数据量;  When the preset persistence condition satisfies 吋, the amount of persistent data of each type of data is determined based on the scoring interval in which the scores of the various types of data in the cache are located, wherein different scoring intervals correspond to different amounts of persistent data;
基于确定的各类数据的持久化数据量对缓存中的各类数据进行持久化  Persistence of various types of data in the cache based on the determined amount of persistent data of various types of data
[权利要求 2] 根据权利要求 1所述的数据处理方法, 其特征在于, 所述评分区间与 持久化数据量成正相关; [Claim 2] The data processing method according to claim 1, wherein the scoring interval is positively correlated with the amount of persistent data;
所述基于当前针对操作的数据所属的类型, 以及预设的与当前监听到 的操作事件的类型关联的计分算法, 更新相应类型的数据的分数, 包 括:  Updating the score of the corresponding type of data based on the type of the current data for the operation and the preset scoring algorithm associated with the type of the currently monitored operation event, including:
若当前监听到的操作事件的类型为査询事件, 则基于当前被操作的数 据所属的类型, 将相应类型的数据的分数加上预设的第一分值, 以更 新相应类型的数据的分数;  If the type of the currently monitored operation event is a query event, based on the type of the currently operated data, the score of the corresponding type of data is added to the preset first score to update the score of the corresponding type of data. ;
若当前监听到的操作事件的类型为插入事件, 则基于当前被操作的数 据所属的类型, 将相应类型的数据的分数加上预设的第二分值, 以更 新相应类型的数据的分数;  If the type of the currently monitored operation event is an insertion event, the score of the corresponding type of data is added to the preset second score based on the type of the currently operated data to update the score of the corresponding type of data;
若当前监听到的操作事件的类型为更新事件, 则基于当前被操作的数 据所属的类型, 将相应类型的数据的分数减去预设的第三分值, 以相 应类型的数据的分数。  If the type of the currently monitored operation event is an update event, the score of the corresponding type of data is subtracted from the preset third score based on the type of the currently operated data, with the score of the corresponding type of data.
[权利要求 3] 根据权利要求 1或 2所述的数据处理方法, 其特征在于, 预先为部分数 据设置高优先级; 所述基于确定的各类数据的持久化数据量对缓存中的各类数据进行持 久化, 包括: [Claim 3] The data processing method according to claim 1 or 2, wherein a high priority is set in advance for part of the data; The persistent data volume based on the determined various types of data persists various types of data in the cache, including:
对高优先级的数据优先进行持久化。  Prioritize high priority data.
[权利要求 4] 根据权利要求 1或 2所述的数据处理方法, 其特征在于, 所述当预设的 持久化条件满足吋, 基于缓存中的各类数据实吋的分数所在的评分区 间, 确定各类数据的持久化数据量, 包括: [Claim 4] The data processing method according to claim 1 or 2, wherein, when the preset persistence condition satisfies 吋, the score interval based on the score of the various types of data in the cache is Determine the amount of persistent data for each type of data, including:
当距离前一次执行所述持久化的吋间间隔达到预设的间隔阈值吋, 基 于各类数据实吋的分数所在的评分区间, 确定各类数据的持久化数据  When the interval between the previous execution of the persistence reaches the preset interval threshold, the persistent data of each type of data is determined based on the score interval in which the scores of the various data are actually calculated.
[权利要求 5] 根据权利要求 1或 2所述的数据处理方法, 其特征在于, 所述数据处理 方法, 还包括: [Claim 5] The data processing method according to claim 1 or 2, wherein the data processing method further comprises:
当数据在缓存中的吋间到达预设的与该数据关联的有效吋间吋, 对达 到所述有效吋间的数据进行持久化。  When the data arrives at the preset effective time associated with the data in the buffer, the data reaching the valid time is persisted.
[权利要求 6] 一种电子设备, 其特征在于, 包括存储器、 处理器及存储在所述存储 器上并可在所述处理器上运行的计算机可读指令, 所述处理器执行所 述计算机可读指令吋实现如下步骤:  [Claim 6] An electronic device, comprising: a memory, a processor, and computer readable instructions stored on the memory and executable on the processor, the processor executing the computer Read the command to achieve the following steps:
监听针对缓存中的各类数据的操作事件;  Listen for operational events for various types of data in the cache;
当监听到针对缓存中的任一类数据的操作事件吋, 基于当前被操作的 数据所属的类型, 以及预设的与当前监听到的操作事件的类型关联的 计分算法, 更新相应类型的数据的分数, 其中, 每类数据对应一分数 , 且各类数据的初始化的分数为预设值;  When an operation event for any type of data in the cache is monitored, the corresponding type of data is updated based on the type of the currently operated data and the preset scoring algorithm associated with the type of the currently monitored operation event. a score, where each type of data corresponds to a score, and the initialized score of each type of data is a preset value;
当预设的持久化条件满足吋, 基于缓存中的各类数据实吋的分数所在 的评分区间, 确定各类数据的持久化数据量, 其中, 不同评分区间对 应不同的持久化数据量;  When the preset persistence condition satisfies 吋, the amount of persistent data of each type of data is determined based on the scoring interval in which the scores of the various types of data in the cache are located, wherein different scoring intervals correspond to different amounts of persistent data;
基于确定的各类数据的持久化数据量对缓存中的各类数据进行持久化  Persistence of various types of data in the cache based on the determined amount of persistent data of various types of data
[权利要求 7] 根据权利要求 6所述的电子设备, 其特征在于, 所述评分区间与持久 化数据量成正相关; 所述基于当前针对操作的数据所属的类型, 以及预设的与当前监听到 的操作事件的类型关联的计分算法, 更新相应类型的数据的分数, 包 括: [Claim 7] The electronic device according to claim 6, wherein the scoring interval is positively correlated with the amount of persistent data; Updating the score of the corresponding type of data based on the type of the current data for the operation and the preset scoring algorithm associated with the type of the currently monitored operation event, including:
若当前监听到的操作事件的类型为査询事件, 则基于当前被操作的数 据所属的类型, 将相应类型的数据的分数加上预设的第一分值, 以更 新相应类型的数据的分数;  If the type of the currently monitored operation event is a query event, based on the type of the currently operated data, the score of the corresponding type of data is added to the preset first score to update the score of the corresponding type of data. ;
若当前监听到的操作事件的类型为插入事件, 则基于当前被操作的数 据所属的类型, 将相应类型的数据的分数加上预设的第二分值, 以更 新相应类型的数据的分数;  If the type of the currently monitored operation event is an insertion event, the score of the corresponding type of data is added to the preset second score based on the type of the currently operated data to update the score of the corresponding type of data;
若当前监听到的操作事件的类型为更新事件, 则基于当前被操作的数 据所属的类型, 将相应类型的数据的分数减去预设的第三分值, 以相 应类型的数据的分数。  If the type of the currently monitored operation event is an update event, the score of the corresponding type of data is subtracted from the preset third score based on the type of the currently operated data, with the score of the corresponding type of data.
[权利要求 8] 根据权利要求 6或 7所述的电子设备, 其特征在于, 预先为部分数据设 置高优先级;  [Claim 8] The electronic device according to claim 6 or 7, wherein a high priority is set in advance for part of the data;
所述基于确定的各类数据的持久化数据量对缓存中的各类数据进行持 久化, 包括:  The persistent data volume based on the determined various types of data persists various types of data in the cache, including:
对高优先级的数据优先进行持久化。  Prioritize high priority data.
[权利要求 9] 根据权利要求 6或 7所述的电子设备, 其特征在于, 所述当预设的持久 化条件满足吋, 基于缓存中的各类数据实吋的分数所在的评分区间, 确定各类数据的持久化数据量, 包括: [Claim 9] The electronic device according to claim 6 or 7, wherein, when the preset persistence condition satisfies 吋, the score interval based on the score of each type of data in the cache is determined. The amount of persistent data for all types of data, including:
当距离前一次执行所述持久化的吋间间隔达到预设的间隔阈值吋, 基 于各类数据实吋的分数所在的评分区间, 确定各类数据的持久化数据  When the interval between the previous execution of the persistence reaches the preset interval threshold, the persistent data of each type of data is determined based on the score interval in which the scores of the various data are actually calculated.
[权利要求 10] 根据权利要求 6或 7所述的电子设备, 其特征在于, 所述处理器执行所 述计算机可读指令吋还实现如下步骤: [Claim 10] The electronic device according to claim 6 or 7, wherein the processor executes the computer readable instructions and further implements the following steps:
当数据在缓存中的吋间到达预设的与该数据关联的有效吋间吋, 对达 到所述有效吋间的数据进行持久化。  When the data arrives at the preset effective time associated with the data in the buffer, the data reaching the valid time is persisted.
[权利要求 11] 一种电子设备, 其特征在于, 包括: 监听模块, 用于监听针对缓存中的各类数据的操作事件; 更新模块, 用于当所述监听模块监听到针对缓存中的任一类数据的操 作事件吋, 基于当前被操作的数据所属的类型, 以及预设的与当前监 听到的操作事件的类型关联的计分算法, 更新相应类型的数据的分数 , 其中, 每类数据对应一分数, 且各类数据的初始化的分数为预设值 确定模块, 用于当预设的持久化条件满足吋, 基于缓存中的各类数据 实吋的分数所在的评分区间, 确定各类数据的持久化数据量, 其中, 不同评分区间对应不同的持久化数据量; [Claim 11] An electronic device, comprising: a monitoring module, configured to listen to an operation event for various types of data in the cache; an update module, configured to: when the listening module listens to an operation event for any type of data in the cache, based on the data currently being operated a type, and a preset scoring algorithm associated with the type of the currently monitored operation event, updating a score of the corresponding type of data, wherein each type of data corresponds to a score, and the initialized score of each type of data is a preset value a determining module, configured to determine a persistent data amount of each type of data based on a scoring interval in which the scores of the various types of data in the cache are located when the preset persistence condition is satisfied, wherein different scoring intervals correspond to different persistent Amount of data
持久化模块, 用于基于所述确定模块确定的各类数据的持久化数据量 对缓存中的各类数据进行持久化。  a persistence module for persisting various types of data in the cache based on the amount of persistent data of various types of data determined by the determining module.
[权利要求 12] 根据权利要求 11所述的电子设备, 其特征在于, 所述评分区间与持久 化数据量成正相关; [Claim 12] The electronic device according to claim 11, wherein the scoring interval is positively correlated with the amount of persistent data;
所述更新模块具体用于: 若所述监听模块当前监听到的操作事件的类 型为査询事件, 则基于当前被操作的数据所属的类型, 将相应类型的 数据的分数加上预设的第一分值, 以更新相应类型的数据的分数; 若 所述监听模块当前监听到的操作事件的类型为插入事件, 则基于当前 被操作的数据所属的类型, 将相应类型的数据的分数加上预设的第二 分值, 以更新相应类型的数据的分数; 若所述监听模块当前监听到的 操作事件的类型为更新事件, 则基于当前被操作的数据所属的类型, 将相应类型的数据的分数减去预设的第三分值, 以相应类型的数据的 分数。  The update module is specifically configured to: if the type of the operation event currently monitored by the monitoring module is a query event, add a score of the corresponding type of data to the preset number based on the type of the currently operated data a score to update the score of the corresponding type of data; if the type of the operation event currently being monitored by the listening module is an insertion event, the score of the corresponding type of data is added based on the type of the currently operated data. a preset second score to update the score of the corresponding type of data; if the type of the operation event currently monitored by the listening module is an update event, the data of the corresponding type is based on the type of the currently operated data The score is subtracted from the preset third score, with the score of the corresponding type of data.
[权利要求 13] 根据权利要求 11或 12所述的电子设备, 其特征在于, 预先为部分数据 设置高优先级;  [Claim 13] The electronic device according to claim 11 or 12, wherein a high priority is set in advance for part of the data;
所述持久化模块具体用于:对高优先级的数据优先进行持久化。  The persistence module is specifically configured to: prioritize high priority data.
[权利要求 14] 根据权利要求 11或 12所述的电子设备, 其特征在于, 所述确定模块具 体用于: 当距离前一次执行所述持久化的吋间间隔达到预设的间隔阈 值吋, 基于各类数据实吋的分数所在的评分区间, 确定各类数据的持 久化数据量。 [Claim 14] The electronic device according to claim 11 or 12, wherein the determining module is specifically configured to: when a distance between the previous execution of the persistence is reached by a preset interval threshold, Based on the scoring interval of the scores of various types of data, determine the holding of various types of data. Long-term data volume.
[权利要求 15] 根据权利要求 11或 12所述的电子设备, 其特征在于, 所述持久化模块 还用于: 当数据在缓存中的吋间到达预设的与该数据关联的有效吋间 吋, 对达到所述有效吋间的数据进行持久化。  [Claim 15] The electronic device according to claim 11 or 12, wherein the persistence module is further configured to: when the data arrives in the cache, the preset effective time associated with the data吋, persisting the data that reaches the effective time.
[权利要求 16] 一种计算机可读存储介质, 所述计算机可读存储介质上存储有计算机 可读指令, 其特征在于, 所述计算机可读指令被处理器执行吋实现以 下步骤:  [Claim 16] A computer readable storage medium having stored thereon computer readable instructions, wherein the computer readable instructions are executed by a processor to implement the following steps:
监听针对缓存中的各类数据的操作事件;  Listen for operational events for various types of data in the cache;
当监听到针对缓存中的任一类数据的操作事件吋, 基于当前被操作的 数据所属的类型, 以及预设的与当前监听到的操作事件的类型关联的 计分算法, 更新相应类型的数据的分数, 其中, 每类数据对应一分数 , 且各类数据的初始化的分数为预设值;  When an operation event for any type of data in the cache is monitored, the corresponding type of data is updated based on the type of the currently operated data and the preset scoring algorithm associated with the type of the currently monitored operation event. a score, where each type of data corresponds to a score, and the initialized score of each type of data is a preset value;
当预设的持久化条件满足吋, 基于缓存中的各类数据实吋的分数所在 的评分区间, 确定各类数据的持久化数据量, 其中, 不同评分区间对 应不同的持久化数据量;  When the preset persistence condition satisfies 吋, the amount of persistent data of each type of data is determined based on the scoring interval in which the scores of the various types of data in the cache are located, wherein different scoring intervals correspond to different amounts of persistent data;
基于确定的各类数据的持久化数据量对缓存中的各类数据进行持久化  Persistence of various types of data in the cache based on the determined amount of persistent data of various types of data
[权利要求 17] 根据权利要求 16所述的计算机可读存储介质, 其特征在于, 所述评分 区间与持久化数据量成正相关; The computer readable storage medium according to claim 16, wherein the score interval is positively correlated with the amount of persistent data;
所述基于当前针对操作的数据所属的类型, 以及预设的与当前监听到 的操作事件的类型关联的计分算法, 更新相应类型的数据的分数, 包 括:  Updating the score of the corresponding type of data based on the type of the current data for the operation and the preset scoring algorithm associated with the type of the currently monitored operation event, including:
若当前监听到的操作事件的类型为査询事件, 则基于当前被操作的数 据所属的类型, 将相应类型的数据的分数加上预设的第一分值, 以更 新相应类型的数据的分数;  If the type of the currently monitored operation event is a query event, based on the type of the currently operated data, the score of the corresponding type of data is added to the preset first score to update the score of the corresponding type of data. ;
若当前监听到的操作事件的类型为插入事件, 则基于当前被操作的数 据所属的类型, 将相应类型的数据的分数加上预设的第二分值, 以更 新相应类型的数据的分数; 若当前监听到的操作事件的类型为更新事件, 则基于当前被操作的数 据所属的类型, 将相应类型的数据的分数减去预设的第三分值, 以相 应类型的数据的分数。 If the type of the currently monitored operation event is an insertion event, the score of the corresponding type of data is added to the preset second score based on the type of the currently operated data to update the score of the corresponding type of data; If the type of the currently monitored operation event is an update event, the score of the corresponding type of data is subtracted from the preset third score based on the type of the currently operated data, with the score of the corresponding type of data.
[权利要求 18] 根据权利要求 16或 17所述的计算机可读存储介质, 其特征在于, 预先 为部分数据设置高优先级;  [Claim 18] The computer readable storage medium according to claim 16 or 17, wherein a high priority is set in advance for the partial data;
所述基于确定的各类数据的持久化数据量对缓存中的各类数据进行持 久化, 包括:  The persistent data volume based on the determined various types of data persists various types of data in the cache, including:
对高优先级的数据优先进行持久化。  Prioritize high priority data.
[权利要求 19] 根据权利要求 16或 17所述的计算机可读存储介质, 其特征在于, 所述 当预设的持久化条件满足吋, 基于缓存中的各类数据实吋的分数所在 的评分区间, 确定各类数据的持久化数据量, 包括: [Claim 19] The computer readable storage medium according to claim 16 or 17, wherein, when the preset persistence condition satisfies 吋, the score based on the score of the various types of data in the cache is Interval, determining the amount of persistent data for each type of data, including:
当距离前一次执行所述持久化的吋间间隔达到预设的间隔阈值吋, 基 于各类数据实吋的分数所在的评分区间, 确定各类数据的持久化数据  When the interval between the previous execution of the persistence reaches the preset interval threshold, the persistent data of each type of data is determined based on the score interval in which the scores of the various data are actually calculated.
[权利要求 20] 根据权利要求 16或 17所述的计算机可读存储介质, 其特征在于, 所述 计算机可读指令被处理器执行吋还实现以下步骤: 当数据在缓存中的 吋间到达预设的与该数据关联的有效吋间吋, 对达到所述有效吋间的 数据进行持久化。 [Claim 20] The computer readable storage medium according to claim 16 or 17, wherein the computer readable instructions are executed by a processor, further implementing the following steps: when the data arrives in the cache The effective time associated with the data is set to persist the data that reaches the effective time.
PCT/CN2017/104534 2017-08-10 2017-09-29 Data processing method, electronic device and computer readable storage medium WO2019028988A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710680232.XA CN107589907B (en) 2017-08-10 2017-08-10 Data processing method, electronic device and computer readable storage medium
CN201710680232.X 2017-08-10

Publications (1)

Publication Number Publication Date
WO2019028988A1 true WO2019028988A1 (en) 2019-02-14

Family

ID=61042027

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/104534 WO2019028988A1 (en) 2017-08-10 2017-09-29 Data processing method, electronic device and computer readable storage medium

Country Status (2)

Country Link
CN (1) CN107589907B (en)
WO (1) WO2019028988A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111211993B (en) * 2018-11-21 2023-08-11 百度在线网络技术(北京)有限公司 Incremental persistence method, device and storage medium for stream computation
CN113127557B (en) * 2019-12-31 2022-12-13 中国移动通信集团四川有限公司 Data persistence method and device based on redis performance and electronic equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102388358A (en) * 2011-09-30 2012-03-21 华为技术有限公司 Method and system for configurating memory device under hybrid storage environment
US20130226909A1 (en) * 2010-10-06 2013-08-29 Hitachi, Ltd. Stream Data Processing Method and Device
CN105824902A (en) * 2016-03-14 2016-08-03 乐视网信息技术(北京)股份有限公司 Data caching method and device
CN105843892A (en) * 2016-03-22 2016-08-10 北京建飞无限科技有限公司 Data persistence method, device and system based on Ehcache

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101510838B (en) * 2009-02-26 2012-07-04 北京北纬点易信息技术有限公司 Method for implementing perdurable data queue
CN101777154A (en) * 2010-02-01 2010-07-14 浪潮集团山东通用软件有限公司 Persistence method of workflow data in workflow management system
US20120179874A1 (en) * 2011-01-07 2012-07-12 International Business Machines Corporation Scalable cloud storage architecture
CN102682110B (en) * 2012-05-10 2013-09-18 北京大学 High-performance cache design method orienting to massive spatial information
CN104317958B (en) * 2014-11-12 2018-01-16 北京国双科技有限公司 A kind of real-time data processing method and system
CN106682193B (en) * 2016-12-29 2021-01-08 北京奇虎科技有限公司 Data persistence storage method and device based on cache

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130226909A1 (en) * 2010-10-06 2013-08-29 Hitachi, Ltd. Stream Data Processing Method and Device
CN102388358A (en) * 2011-09-30 2012-03-21 华为技术有限公司 Method and system for configurating memory device under hybrid storage environment
CN105824902A (en) * 2016-03-14 2016-08-03 乐视网信息技术(北京)股份有限公司 Data caching method and device
CN105843892A (en) * 2016-03-22 2016-08-10 北京建飞无限科技有限公司 Data persistence method, device and system based on Ehcache

Also Published As

Publication number Publication date
CN107589907A (en) 2018-01-16
CN107589907B (en) 2019-12-13

Similar Documents

Publication Publication Date Title
CN109873904B (en) User message state reporting processing method, device and storage medium for message pushing
US8498966B1 (en) Systems and methods for adaptively performing backup operations
WO2020140614A1 (en) Offline message distribution method, server and storage medium
CN105574141B (en) Method and device for carrying out data migration on database
WO2017004952A1 (en) Method and apparatus for detecting and stopping malicious clicking of advertisement link
CN108089825B (en) Storage system based on distributed cluster
CN107766160B (en) queue message processing method and terminal equipment
US20210192046A1 (en) Resource Management Unit for Capturing Operating System Configuration States and Managing Malware
US11782761B2 (en) Resource management unit for capturing operating system configuration states and offloading tasks
CN111143158A (en) Monitoring data real-time storage method and system, electronic equipment and storage medium
CN111813573A (en) Communication method of management platform and robot software and related equipment thereof
WO2019028988A1 (en) Data processing method, electronic device and computer readable storage medium
US8060773B1 (en) Systems and methods for managing sub-clusters within a multi-cluster computing system subsequent to a network-partition event
CN103347031B (en) A kind of method and apparatus taking precautions against ARP message aggression
WO2014114089A1 (en) Method and system for optimizing load balance of distributed file system
WO2019028987A1 (en) Data processing method, electronic device and computer readable storage medium
WO2023185335A1 (en) Crash clustering method and apparatus, electronic device and storage medium
US20240095019A1 (en) Out-of-band updating method and system of expander
CN110162563B (en) Data warehousing method and system, electronic equipment and storage medium
CN111858393A (en) Memory page management method, memory page management device, medium and electronic device
CN102006285B (en) A kind of message processing method for Network Security Device and device
WO2023206963A1 (en) Data processing method and system, and related components
CN111625350A (en) Memory allocation method, device, equipment and storage medium for network message data
CN112463037B (en) Metadata storage method, device, equipment and product
CN109426563B (en) Process management method and device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17921238

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 25.06.2020)

122 Ep: pct application non-entry in european phase

Ref document number: 17921238

Country of ref document: EP

Kind code of ref document: A1