CN111198793A - Data deleting method and system, medium and computer system - Google Patents
Data deleting method and system, medium and computer system Download PDFInfo
- Publication number
- CN111198793A CN111198793A CN201811381811.5A CN201811381811A CN111198793A CN 111198793 A CN111198793 A CN 111198793A CN 201811381811 A CN201811381811 A CN 201811381811A CN 111198793 A CN111198793 A CN 111198793A
- Authority
- CN
- China
- Prior art keywords
- task
- information
- cluster
- deleted
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000012217 deletion Methods 0.000 claims abstract description 75
- 230000037430 deletion Effects 0.000 claims abstract description 75
- 230000015654 memory Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 10
- 238000012544 monitoring process Methods 0.000 description 9
- 238000012545 processing Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000004083 survival effect Effects 0.000 description 3
- 238000004140 cleaning Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 239000000758 substrate Substances 0.000 description 2
- 108010001267 Protein Subunits Proteins 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Stored Programmes (AREA)
Abstract
The present disclosure provides a data deletion method, including: acquiring configuration information, wherein the configuration information is at least used for representing data needing to be deleted; generating at least one delete task based on the configuration information; and distributing at least one deleting task to the corresponding client, wherein the client can execute the deleting task and delete the data which is represented by the configuration information and needs to be deleted. The present disclosure also provides a data deletion system, a computer system and a computer-readable storage medium.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and more particularly, to a data deletion method, a data deletion system, a computer system, and a computer-readable storage medium.
Background
In the big data era, data storage is of considerable importance. The data storage based on the elastic search (hereinafter referred to as ES) can quickly respond to the query, and simultaneously supports high concurrent writing, but the stored data is more and more, and the expired data needs to be cleaned up regularly as time goes on. The ES 1.7 version supports the TTL cleaning mode, but the ES 2.0 version and the versions above do not support the cleaning mode any more, and users need to do processing according to actual conditions. In this regard, most users currently employ development timing tasks to periodically delete stale data.
In implementing the disclosed concept, the inventors found the following drawbacks in the related art: the development of the timing task requires that a user manually writes worker deletion data based on hard codes for each service, and dynamic adjustment is not convenient.
Disclosure of Invention
In view of the above, the present disclosure provides a data deletion method and system capable of automatically generating a data deletion task based on configuration information of a user.
One aspect of the present disclosure provides a data deletion method, including: acquiring configuration information, wherein the configuration information is at least used for representing data needing to be deleted; generating at least one deletion task based on the configuration information; and distributing the at least one deleting task to corresponding terminal equipment, wherein the terminal equipment can execute the deleting task and delete the data which is represented by the configuration information and needs to be deleted.
According to an embodiment of the present disclosure, the generating at least one deletion task based on the configuration information includes: analyzing the configuration information, and determining the following information of the data needing to be deleted: the name of the ES cluster and the IP address of the node used for storing the data needing to be deleted in the ES cluster; and generating at least one deletion task capable of deleting data corresponding to the information and having expired currently, based on the determined information.
According to an embodiment of the present disclosure, the generating at least one deletion task based on the configuration information includes: analyzing the configuration information, and determining the following information of the data needing to be deleted: the method comprises the following steps of obtaining the name of an ES cluster, the IP address of a node used for storing data needing to be deleted in the ES cluster and a specified time parameter; and generating at least one deletion task capable of deleting data corresponding to the information and having a lifetime value less than the specified time parameter, based on the determined information.
According to an embodiment of the present disclosure, the method further includes: acquiring state information of the ES cluster; generating a task adjusting instruction according to the acquired state information; and adjusting the execution pace of the at least one deleted task on the corresponding terminal device through the task adjusting instruction.
According to an embodiment of the present disclosure, the status information includes one or more of the following information of the ES cluster: load state, memory state, CPU state, and disk state.
According to an embodiment of the present disclosure, the allocating the at least one deletion task to the corresponding terminal device includes: determining the number of available terminal devices and the speed of executing tasks, and the number of the at least one deleting task; and allocating the at least one deletion task to the available terminal device according to the determined number and speed.
Another aspect of the present disclosure provides a data deletion system, including: the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring configuration information, and the configuration information is at least used for representing data needing to be deleted; the first generation module is used for generating at least one deletion task based on the configuration information; and an allocation module, configured to allocate the at least one deletion task to a corresponding terminal device, where the terminal device is capable of executing the deletion task and deleting the data that needs to be deleted and is represented by the configuration information.
According to an embodiment of the present disclosure, the first generating module includes: a first analyzing unit, configured to analyze the configuration information, and determine the following information of the data that needs to be deleted: the name of the ES cluster and the IP address of the node used for storing the data needing to be deleted in the ES cluster; and a first generation unit configured to generate, based on the determined information, at least one deletion task capable of deleting data corresponding to the information and having expired currently.
According to an embodiment of the present disclosure, the first generating module includes: a second analyzing unit, configured to analyze the configuration information, and determine the following information of the data that needs to be deleted: the method comprises the following steps of obtaining the name of an ES cluster, the IP address of a node used for storing data needing to be deleted in the ES cluster and a specified time parameter; and a second generation unit configured to generate, based on the determined information, at least one deletion task capable of deleting data corresponding to the information and having a lifetime value smaller than the specified time parameter.
According to an embodiment of the present disclosure, the above system further includes: a second obtaining module, configured to obtain state information of the ES cluster; the second generation module is used for generating a task adjusting instruction according to the acquired state information; and the adjusting module is used for adjusting the execution pace of the at least one deleted task on the corresponding terminal equipment through the task adjusting instruction.
According to an embodiment of the present disclosure, the status information includes one or more of the following information of the ES cluster: load state, memory state, CPU state, and disk state.
According to an embodiment of the present disclosure, the allocation module includes: a determining unit, configured to determine the number of available terminal devices and the speed of executing the task, and the number of the at least one deletion task; and an allocation unit for allocating the at least one deletion task to the available terminal device according to the determined number and speed.
Another aspect of the present disclosure provides a computer system comprising: one or more processors; memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement a method as described in any above.
Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions for implementing the method as described above when executed.
Another aspect of the disclosure provides a computer program comprising computer executable instructions for implementing the method as described above when executed.
According to the embodiment of the disclosure, because a technical means of setting a Web UI interface for a user to input configuration information and further enabling a client to automatically generate a data deleting task based on the configuration information output by the user is adopted, the technical problem that in the related art, when data stored in an ES is deleted, a user needs to manually write a hard coding timing task for each service, so that dynamic adjustment is inconvenient is at least partially solved, and further, the technical effect that the corresponding deleting task can be automatically generated for each service, so that dynamic adjustment is facilitated is achieved.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:
FIG. 1 schematically illustrates an exemplary system architecture to which the data deletion methods and systems of the present disclosure may be applied;
FIG. 2 schematically illustrates a flow diagram of a data deletion method according to an embodiment of the disclosure;
3A-3B schematically illustrate the composition of configuration information according to an embodiment of the disclosure;
fig. 4 schematically illustrates composition of state information of an ES cluster according to an embodiment of the present disclosure;
FIG. 5 schematically illustrates a diagram for dynamically adjusting the execution cadence of tasks, in accordance with an embodiment of the disclosure;
FIG. 6 schematically illustrates a block diagram of a data deletion system according to an embodiment of the disclosure; and
FIG. 7 schematically illustrates a block diagram of a computer system suitable for implementing the data deletion method and system according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a convention analogous to "A, B or at least one of C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B or C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
The embodiment of the disclosure provides a data deleting method and system capable of automatically generating a data deleting task based on configuration information of a user. The method comprises the steps of obtaining configuration information, wherein the configuration information is at least used for representing data needing to be deleted; generating at least one delete task based on the configuration information; and distributing at least one deleting task to the corresponding client, wherein the client can execute the deleting task and delete the data which is represented by the configuration information and needs to be deleted.
Fig. 1 schematically illustrates an exemplary system architecture to which the data deletion methods and systems of the present disclosure may be applied. It should be noted that fig. 1 is only an example of a system architecture to which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, and does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios.
As shown in fig. 1, the system architecture according to this embodiment may include an ES cluster, a WEB-UI configuration center, a task generator, terminals 1 and 2, a monitoring center, and a coordinator.
An ES cluster is used to store data and may include a plurality of nodes, each having its own IP address. And the WEB-UI configuration center is used for providing a WEB-UI interface, and a user can input configuration information through the WEB-UI interface. The task generator may be a client, a server or a server cluster, and is configured to automatically generate a deletion task according to the configuration information, and distribute the deletion task to corresponding terminal devices, such as the terminal 1 and the terminal 2, for execution. The monitoring center can be a client, a server or a server cluster and is used for monitoring the state of the ES cluster and sending the monitored state information to the coordinator. The coordinator can be a client, a server or a server cluster and is used for coordinating the execution pace of the deletion task according to the state information of the ES cluster.
It should be noted that the WEB-UI configuration center, the task generator, the monitoring center, and the coordinator may be a device, and specifically, may be a device having an information configuration function, a task generation function, a monitoring function, and a coordination function at the same time. Of course, the WEB-UI configuration center, the task generator, the monitoring center and the coordinator may also be four different devices.
It should be noted that the data deleting method provided by the embodiment of the present disclosure may be generally executed by a server/server cluster. Accordingly, the data deletion system provided by the embodiments of the present disclosure may be generally disposed in a server/server cluster. Alternatively, the data deleting method provided by the embodiment of the present disclosure may also be executed by the terminal device. Correspondingly, the data deleting system provided by the embodiment of the disclosure can also be arranged in the terminal device.
It should be understood that the number of ES clusters, WEB-UI configuration centers, task generators, terminal devices, monitoring centers, coordinators in fig. 1 are merely illustrative. Any number of ES clusters, WEB-UI configuration centers, task generators, terminal devices, monitoring centers, and coordinators may be provided, as desired for implementation.
Fig. 2 schematically shows a flow chart of a data deletion method according to an embodiment of the present disclosure.
As shown in fig. 2, the method includes operations S210 to S230, in which:
in operation S210, configuration information is obtained, wherein the configuration information is at least used for characterizing data that needs to be deleted.
Specifically, a user may input configuration information such as the name of the ES cluster and the IP address of any one of the nodes in a configuration table through a WEB-UI configuration center as shown in fig. 1, and a task generator may periodically scan the configuration table input by the user through a timing task and read meta information of the ES cluster (including the name of the ES cluster and the IP address of the relevant node therein) according to the configuration information.
In operation S220, at least one deletion task is generated based on the configuration information.
Specifically, as shown in fig. 1, assuming that the user inputs configuration information shown in table 1 into the configuration table, the task generator may automatically generate two deletion tasks according to the information, namely, task1 and task2, where the execution task1 may delete related data on the node 1 corresponding to the IP1 address in the ES _ name1 cluster, and the execution task2 may delete related data on the node 2 corresponding to the IP2 address in the ES _ name1 cluster.
TABLE 1
| Name of ES cluster | IP of a node | Time parameter |
| ES_namel | IP1 | |
| ES_namel | IP2 |
It should be noted that, in the embodiment of the present disclosure, the task generator may specifically automatically generate the delete task by using a dynamically generated bytecode technology (mainly java assit, CGLIB, and the like).
In operation S230, at least one deletion task is assigned to a corresponding terminal device, where the terminal device is capable of executing the deletion task and deleting the data to be deleted, which is represented by the configuration information.
Specifically, as shown in fig. 1, the task generator may randomly allocate the deletion task to any one or more terminal devices in the system architecture for processing. For example, task1 may be assigned to terminal 1, task2 may be assigned to terminal 2, or both task1 and task2 may be assigned to terminal 1, or both task1 and task2 may be assigned to terminal 2.
When the terminal device executes task1, the terminal device can operate the node 1 corresponding to the IP1 address in the ES _ name1 cluster, so as to delete the related data stored in the node 1; when the terminal device executes task2, the terminal device can operate the node 2 corresponding to the IP2 address in the ES _ name2 cluster, so as to delete the related data stored in the node 2.
Compared with the prior art that when data stored in an ES cluster is deleted, a user needs to manually write a hard-coded timed deletion task for each service, and dynamic adjustment of the tasks is inconvenient.
The method of fig. 2 is further described with reference to fig. 3A-3B, and fig. 4-5, in conjunction with specific embodiments.
As an alternative embodiment, generating at least one deletion task based on the configuration information includes: analyzing the configuration information, and determining the following information of the data needing to be deleted: the name of the ES cluster and the IP address of the node in the ES cluster for storing the data needing to be deleted; and generating at least one deletion task capable of deleting data corresponding to the information and having expired currently, based on the determined information.
As shown in fig. 3A, the configuration information may include only the name of the ES cluster, and the IP address of the associated node in the ES cluster. In other words, the user may not configure the time parameter. In this case, the task generator sets the time parameter as a default parameter when generating the delete task, i.e. deletes by default the data that has currently expired.
Through the embodiment of the disclosure, the user can only specify the deletion position (i.e. the name of the ES cluster and the IP address of the relevant node therein) when configuring the information, and does not specify which data is to be specifically deleted (i.e. does not specify the time parameter), and at this time, the data that has expired currently can be deleted by default according to the rule.
As an alternative embodiment, generating at least one deletion task based on the configuration information includes: analyzing the configuration information, and determining the following information of the data needing to be deleted: the method comprises the following steps of (1) obtaining the name of an ES cluster, the IP address of a node used for storing data needing to be deleted in the ES cluster and a specified time parameter; and generating at least one deletion task capable of deleting data corresponding to the information and having a lifetime value less than the specified time parameter, based on the determined information.
Slightly different from the configuration information shown in fig. 3A, the configuration information may include a user-specified time parameter in addition to the name of the ES cluster and the IP address of the associated node in the ES cluster, as shown in fig. 3B. In other words, the user may configure the time parameter. In this case, the task generator sets the time parameter to the user-specified time parameter when generating the deletion task, that is, deletes data whose survival time is earlier than the time parameter by default.
Through the embodiment of the disclosure, the user can specify the deletion position (i.e. the name of the ES cluster and the IP address of the relevant node therein) and specify which data are specifically deleted (i.e. specify the time parameter) when configuring the information, and at this time, the data with the survival time earlier than the time parameter can be deleted by default according to the rule.
As an alternative embodiment, the method further comprises: acquiring state information of an ES cluster; generating a task adjusting instruction according to the acquired state information; and adjusting the execution pace of at least one deleted task on the corresponding terminal equipment through the task adjusting instruction.
Specifically, as shown in fig. 1, the monitoring center may capture the state of the ES cluster in real time, where the state includes one or more states of a load (e.g., an upper limit value of the load), a memory (e.g., a usage rate of the memory), a CPU (e.g., a usage rate of the CPU), and a disk (e.g., a usage rate of the disk) of the entire cluster (i.e., as shown in fig. 4, the state information may include one or more of the following information of the ES cluster, i.e., the load state, the memory state, the CPU state, and the disk state).
As shown in fig. 5, the coordinator may dynamically read the data of the monitoring center, so as to dynamically adjust the execution pace of each deleted task according to the state of the ES cluster.
For example, if the monitoring finds that the load of the ES cluster reaches the upper limit value, the execution of the delete task may be immediately terminated, and the delete task may be started after the ES cluster is stabilized, thereby reducing the consumption of the cluster load.
As another example, if the monitoring finds that the load of the ES cluster is at a medium level but does not exceed the upper limit, the execution pace of the delete task may be dynamically adjusted to reduce the cluster load.
For another example, if the ES cluster load is found to be low by monitoring, the execution pace of the deleter task may be dynamically increased to improve the deletion efficiency of the data.
Compared with the prior art that the state of the ES cluster cannot be monitored when the tasks are deleted, and the execution pace of each deleted task cannot be dynamically adjusted according to the pressure of the ES cluster, the embodiment of the disclosure can automatically visually monitor the state of the ES cluster in real time and dynamically adjust the execution pace of each deleted task according to the monitoring result.
As an optional embodiment, allocating at least one deletion task to a corresponding terminal device includes: determining the number of available terminal devices and the speed of executing tasks, and the number of at least one deletion task; and allocating the at least one deletion task to the available terminal devices according to the determined number and speed.
Specifically, in conjunction with fig. 1 and 5, the task generator may be able to dynamically allocate tasks according to the number of terminal devices that perform the deletion task, and the number of deletion tasks that are initiated. For example, suppose that there are 2 clusters each having 2 indexes to be deleted and 3 terminal devices executing tasks, at this time, the task generator starts 4 deleted tasks, and the coordinator is responsible for allocating the 4 tasks to the 3 terminal devices to execute, for example, one terminal device may execute 2 tasks, and the other two terminal devices may each execute 1 task. If a certain task is executed quickly, 2 tasks are allocated to another idle terminal device to be executed when the next round of automatic balancing is performed.
By the embodiment of the disclosure, the tasks can be dynamically adjusted, allocated and deleted according to the state of the task execution end and the number of the started tasks.
FIG. 6 schematically shows a block diagram of a data deletion system according to an embodiment of the disclosure.
As shown in FIG. 6, the data deletion system 600 includes a first acquisition module 610, a first generation module 620, and an assignment module 630.
A first obtaining module 610, configured to obtain configuration information, where the configuration information is at least used to represent data that needs to be deleted;
a first generating module 620 for generating at least one delete task based on the configuration information; and
an allocating module 630, configured to allocate at least one deletion task to a corresponding terminal device, where the terminal device can execute the deletion task and delete the data that needs to be deleted and is represented by the configuration information.
Compared with the prior art that when data stored in an ES cluster is deleted, a user needs to manually write a hard-coded timed deletion task for each service, and dynamic adjustment of the tasks is inconvenient.
As an alternative embodiment, the first generating module includes: a first analyzing unit, configured to analyze the configuration information, and determine the following information of the data that needs to be deleted: the name of the ES cluster and the IP address of the node used for storing the data needing to be deleted in the ES cluster; and a first generation unit configured to generate, based on the determined information, at least one deletion task capable of deleting data corresponding to the information and having expired currently.
Through the embodiment of the disclosure, the user can only specify the deletion position (i.e. the name of the ES cluster and the IP address of the relevant node therein) when configuring the information, and does not specify which data is to be specifically deleted (i.e. does not specify the time parameter), and at this time, the data that has expired currently can be deleted by default according to the rule.
As an alternative embodiment, the first generating module includes: a second analyzing unit, configured to analyze the configuration information, and determine the following information of the data that needs to be deleted: the method comprises the following steps of obtaining the name of an ES cluster, the IP address of a node used for storing data needing to be deleted in the ES cluster and a specified time parameter; and a second generation unit configured to generate, based on the determined information, at least one deletion task capable of deleting data corresponding to the information and having a lifetime value smaller than the specified time parameter.
Through the embodiment of the disclosure, the user can specify the deletion position (i.e. the name of the ES cluster and the IP address of the relevant node therein) and specify which data are specifically deleted (i.e. specify the time parameter) when configuring the information, and at this time, the data with the survival time earlier than the time parameter can be deleted by default according to the rule.
As an optional embodiment, the system further includes: a second obtaining module, configured to obtain state information of the ES cluster; the second generation module is used for generating a task adjusting instruction according to the acquired state information; and the adjusting module is used for adjusting the execution pace of the at least one deleted task on the corresponding terminal equipment through the task adjusting instruction.
As an optional embodiment, the status information includes one or more of the following information of the ES cluster: load state, memory state, CPU state, and disk state.
Compared with the prior art that the state of the ES cluster cannot be monitored when the tasks are deleted, and the execution pace of each deleted task cannot be dynamically adjusted according to the pressure of the ES cluster, the embodiment of the disclosure can automatically visually monitor the state of the ES cluster in real time and dynamically adjust the execution pace of each deleted task according to the monitoring result.
As an alternative embodiment, the distribution module includes: a determining unit, configured to determine the number of available terminal devices and the speed of executing the task, and the number of the at least one deletion task; and an allocation unit for allocating the at least one deletion task to the available terminal device according to the determined number and speed.
By the embodiment of the disclosure, the tasks can be dynamically adjusted, allocated and deleted according to the state of the task execution end and the number of the started tasks.
Any of the modules, units, or at least part of the functionality of any of them according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules and units according to the embodiments of the present disclosure may be implemented by being split into a plurality of modules. Any one or more of the modules, units according to the embodiments of the present disclosure may be implemented at least partially as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by any other reasonable means of hardware or firmware by integrating or packaging the circuits, or in any one of three implementations of software, hardware and firmware, or in any suitable combination of any of them. Alternatively, one or more of the modules, units according to embodiments of the present disclosure may be implemented at least partly as computer program modules, which, when executed, may perform the respective functions.
For example, any plurality of the first obtaining module 610, the first generating module 620 and the allocating module 630 may be combined and implemented in one module/unit/sub-unit, or any one of the modules/units/sub-units may be split into a plurality of modules/units/sub-units. Alternatively, at least part of the functionality of one or more of these modules/units/sub-units may be combined with at least part of the functionality of other modules/units/sub-units and implemented in one module/unit/sub-unit. According to an embodiment of the present disclosure, at least one of the first obtaining module 610, the first generating module 620, and the allocating module 630 may be at least partially implemented as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by hardware or the same hardware in any other reasonable manner of integrating or packaging a circuit, or implemented in any one of three implementations of software, hardware, and firmware, or in a suitable combination of any of them. Alternatively, at least one of the first obtaining module 610, the first generating module 620 and the allocating module 630 may be at least partially implemented as a computer program module, which when executed may perform the corresponding functions.
It should be noted that, a system part in the embodiment of the present disclosure corresponds to a method part in the embodiment of the present disclosure, and the description of the system part specifically refers to the method part, which is not described herein again.
FIG. 7 schematically illustrates a block diagram of a computer system suitable for implementing the data deletion method and system according to an embodiment of the present disclosure. The computer system illustrated in FIG. 7 is only one example and should not impose any limitations on the scope of use or functionality of embodiments of the disclosure.
As shown in fig. 7, a computer system 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. The processor 701 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 701 may also include on-board memory for caching purposes. The processor 701 may comprise a single processing unit or a plurality of processing units for performing the different actions of the method flows according to embodiments of the present disclosure.
In the RAM 703, various programs and data necessary for the operation of the system 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other by a bus 704. The processor 701 performs various operations of the method flows according to the embodiments of the present disclosure by executing programs in the ROM 702 and/or the RAM 703. It is noted that the programs may also be stored in one or more memories other than the ROM 702 and RAM 703. The processor 701 may also perform various operations of method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the present disclosure, the system 700 may also include an input/output (I/O) interface 705, the input/output (I/O) interface 705 also being connected to the bus 704. The system 700 may also include one or more of the following components connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.
According to embodiments of the present disclosure, method flows according to embodiments of the present disclosure may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711. The computer program, when executed by the processor 701, performs the above-described functions defined in the system of the embodiment of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples may include, but are not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM 702 and/or the RAM 703 and/or one or more memories other than the ROM 702 and the RAM 703 described above.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.
The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.
Claims (14)
1. A method of data deletion, comprising:
acquiring configuration information, wherein the configuration information is at least used for representing data needing to be deleted;
generating at least one delete task based on the configuration information; and
and distributing the at least one deleting task to corresponding terminal equipment, wherein the terminal equipment can execute the deleting task and delete the data which are represented by the configuration information and need to be deleted.
2. The method of claim 1, wherein the generating at least one delete task based on the configuration information comprises:
analyzing the configuration information, and determining the following information of the data needing to be deleted: the name of the ES cluster and the IP address of the node used for storing the data needing to be deleted in the ES cluster; and
based on the determined information, at least one deletion task capable of deleting data corresponding to the information and that has currently expired is generated.
3. The method of claim 1, wherein the generating at least one delete task based on the configuration information comprises:
analyzing the configuration information, and determining the following information of the data needing to be deleted: the method comprises the following steps of obtaining the name of an ES cluster, the IP address of a node used for storing data needing to be deleted in the ES cluster and a specified time parameter; and
and generating at least one deletion task capable of deleting data corresponding to the information and having a lifetime value less than the specified time parameter, based on the determined information.
4. The method of claim 2 or 3, wherein the method further comprises:
acquiring state information of the ES cluster;
generating a task adjusting instruction according to the acquired state information; and
and adjusting the execution pace of the at least one deleted task on the corresponding terminal equipment through the task adjusting instruction.
5. The method of claim 4, wherein the state information comprises one or more of the following information of the ES cluster: load state, memory state, CPU state, and disk state.
6. The method of claim 1, wherein the assigning the at least one deletion task to a corresponding terminal device comprises:
determining the number of available terminal devices and the speed of executing tasks, and the number of the at least one deleting task; and
assigning the at least one deletion task to the available terminal device according to the determined number and speed.
7. A data deletion system, comprising:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring configuration information, and the configuration information is at least used for representing data needing to be deleted;
a first generating module, configured to generate at least one deletion task based on the configuration information; and
and the distribution module is used for distributing the at least one deleting task to the corresponding terminal equipment, wherein the terminal equipment can execute the deleting task and delete the data which is represented by the configuration information and needs to be deleted.
8. The system of claim 7, wherein the first generation module comprises:
a first parsing unit, configured to parse the configuration information, and determine the following information of the data that needs to be deleted: the name of the ES cluster and the IP address of the node used for storing the data needing to be deleted in the ES cluster; and
a first generating unit configured to generate, based on the determined information, at least one deletion task capable of deleting data corresponding to the information and having expired currently.
9. The system of claim 7, wherein the first generation module comprises:
a second parsing unit, configured to parse the configuration information, and determine the following information of the data that needs to be deleted: the method comprises the following steps of obtaining the name of an ES cluster, the IP address of a node used for storing data needing to be deleted in the ES cluster and a specified time parameter; and
and a second generation unit configured to generate, based on the determined information, at least one deletion task capable of deleting data corresponding to the information and having a lifetime value smaller than the specified time parameter.
10. The system of claim 8 or 9, wherein the system further comprises:
a second obtaining module, configured to obtain state information of the ES cluster;
the second generation module is used for generating a task adjusting instruction according to the acquired state information; and
and the adjusting module is used for adjusting the execution pace of the at least one deleting task on the corresponding terminal equipment through the task adjusting instruction.
11. The system of claim 10, wherein the state information includes one or more of the following information of the ES cluster: load state, memory state, CPU state, and disk state.
12. The system of claim 7, wherein the assignment module comprises:
a determination unit for determining the number of available terminal devices and the speed of executing tasks, and the number of the at least one deletion task; and
an allocation unit for allocating the at least one deletion task to the available terminal device according to the determined number and speed.
13. A computer system, comprising:
one or more processors;
a memory for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-6.
14. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to carry out the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811381811.5A CN111198793B (en) | 2018-11-19 | 2018-11-19 | Data deleting method and system, medium and computer system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811381811.5A CN111198793B (en) | 2018-11-19 | 2018-11-19 | Data deleting method and system, medium and computer system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111198793A true CN111198793A (en) | 2020-05-26 |
| CN111198793B CN111198793B (en) | 2024-10-18 |
Family
ID=70746855
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811381811.5A Active CN111198793B (en) | 2018-11-19 | 2018-11-19 | Data deleting method and system, medium and computer system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111198793B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113395369A (en) * | 2021-06-10 | 2021-09-14 | 北京天融信网络安全技术有限公司 | Cache management method and device, electronic equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101000614A (en) * | 2006-01-11 | 2007-07-18 | 富士通株式会社 | File delete method and file open method |
| CN103874051A (en) * | 2012-12-12 | 2014-06-18 | 中国联合网络通信集团有限公司 | Internet of Things preset resource processing method and device |
| CN106126708A (en) * | 2016-06-30 | 2016-11-16 | 乐视控股（北京）有限公司 | A kind of method and apparatus of the file deleting terminal storage |
| US20170017671A1 (en) * | 2012-04-25 | 2017-01-19 | International Business Machines Corporation | Deleting data in a dispersed storage network |
| CN106357811A (en) * | 2016-10-25 | 2017-01-25 | 广东欧珀移动通信有限公司 | Backup data deleting method, backup data deleting device and backup data deleting system |
| CN106569851A (en) * | 2016-10-21 | 2017-04-19 | 青岛海信电器股份有限公司 | Application processing method and device |
| CN107659929A (en) * | 2017-09-20 | 2018-02-02 | 北京佰才邦技术有限公司 | One kind configuration delet method, system, terminal device and serving BS |
| CN108073355A (en) * | 2016-11-15 | 2018-05-25 | 杭州海康威视数字技术股份有限公司 | A kind of data storage and delet method and device |
-
2018
- 2018-11-19 CN CN201811381811.5A patent/CN111198793B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101000614A (en) * | 2006-01-11 | 2007-07-18 | 富士通株式会社 | File delete method and file open method |
| US20170017671A1 (en) * | 2012-04-25 | 2017-01-19 | International Business Machines Corporation | Deleting data in a dispersed storage network |
| CN103874051A (en) * | 2012-12-12 | 2014-06-18 | 中国联合网络通信集团有限公司 | Internet of Things preset resource processing method and device |
| CN106126708A (en) * | 2016-06-30 | 2016-11-16 | 乐视控股（北京）有限公司 | A kind of method and apparatus of the file deleting terminal storage |
| CN106569851A (en) * | 2016-10-21 | 2017-04-19 | 青岛海信电器股份有限公司 | Application processing method and device |
| CN106357811A (en) * | 2016-10-25 | 2017-01-25 | 广东欧珀移动通信有限公司 | Backup data deleting method, backup data deleting device and backup data deleting system |
| CN108073355A (en) * | 2016-11-15 | 2018-05-25 | 杭州海康威视数字技术股份有限公司 | A kind of data storage and delet method and device |
| CN107659929A (en) * | 2017-09-20 | 2018-02-02 | 北京佰才邦技术有限公司 | One kind configuration delet method, system, terminal device and serving BS |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113395369A (en) * | 2021-06-10 | 2021-09-14 | 北京天融信网络安全技术有限公司 | Cache management method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111198793B (en) | 2024-10-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11231955B1 (en) | Dynamically reallocating memory in an on-demand code execution system | |
| US10942760B2 (en) | Predictive rightsizing for virtual machines in cloud computing systems | |
| US7266823B2 (en) | Apparatus and method of dynamically repartitioning a computer system in response to partition workloads | |
| US20180373540A1 (en) | Cluster graphical processing unit (gpu) resource sharing efficiency by directed acyclic graph (dag) generation | |
| US20140325521A1 (en) | Method and System for Allocating Resources to Tasks in a Build Process | |
| US20140047095A1 (en) | System and method for tuning a cloud computing system | |
| US20140047084A1 (en) | System and method for modifying a hardware configuration of a cloud computing system | |
| US10356167B1 (en) | Workload profiling | |
| CN113535363A (en) | Task calling method and device, electronic equipment and storage medium | |
| CN105786603B (en) | Distributed high-concurrency service processing system and method | |
| US10824339B1 (en) | Snapshot-based garbage collection in an on-demand code execution system | |
| US9471387B2 (en) | Scheduling in job execution | |
| US10048987B1 (en) | Methods and apparatus for a resource sharing platform having resource quality estimation | |
| CN107251007B (en) | Cluster computing service ensuring device and method | |
| CN115145695B (en) | Resource scheduling method and device, computer equipment and storage medium | |
| CN111078516A (en) | Distributed performance test method and device and electronic equipment | |
| US9672073B2 (en) | Non-periodic check-pointing for fine granular retry of work in a distributed computing environment | |
| CN111198793B (en) | Data deleting method and system, medium and computer system | |
| CN104700255B (en) | Multi-process processing method, device and system | |
| Benini et al. | Resource management policy handling multiple use-cases in mpsoc platforms using constraint programming | |
| CN111162973A (en) | Data flow acquisition method and device, electronic equipment and medium | |
| CN116107753A (en) | Task node distribution method and device, electronic equipment and storage medium | |
| CN108197005B (en) | Method, medium, equipment and system for monitoring bottom layer operation performance of IOS application | |
| US20190250957A1 (en) | System and method of dynamic allocation of hardware accelerator | |
| CN113377539A (en) | Processing method and device for realizing load balance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |