CN113515516A - Data processing method and device, computer equipment and computer storage medium - Google Patents

Data processing method and device, computer equipment and computer storage medium Download PDF

Info

Publication number
CN113515516A
CN113515516A CN202110910193.4A CN202110910193A CN113515516A CN 113515516 A CN113515516 A CN 113515516A CN 202110910193 A CN202110910193 A CN 202110910193A CN 113515516 A CN113515516 A CN 113515516A
Authority
CN
China
Prior art keywords
task
data
state
user
data processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110910193.4A
Other languages
Chinese (zh)
Inventor
郑海中
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing ByteDance Network Technology Co Ltd
Original Assignee
Beijing ByteDance Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing ByteDance Network Technology Co Ltd filed Critical Beijing ByteDance Network Technology Co Ltd
Priority to CN202110910193.4A priority Critical patent/CN113515516A/en
Publication of CN113515516A publication Critical patent/CN113515516A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/215Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1446Point-in-time backing up or restoration of persistent data
    • G06F11/1448Management of the data involved in backup or backup restore
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60Protecting data
    • G06F21/602Providing cryptographic facilities or services
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0652Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/546Message passing systems or structures, e.g. queues

Abstract

The present disclosure provides a data processing method, apparatus, computer device, and computer storage medium, wherein the method comprises: acquiring a first data processing request for adjusting the data state of user data sent by a user side; acquiring a first task state of a first task corresponding to the first data processing request at the current moment, wherein the first task is a task for processing user data, and the task state of the first task is used for indicating the data state of the user data; under the condition that the user data meet preset conditions based on the first task state, the task state of the first task is adjusted to be a second task state, and the second task state is stored and sent to the user side; and determining a data processing type corresponding to the second task state, and executing the first task based on the data processing type to adjust the data state of the user data.

Description

Data processing method and device, computer equipment and computer storage medium
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a data processing method, an apparatus, a computer device, and a computer storage medium.
Background
As business functions provided by enterprises become more powerful, the amount of user data generated in the using process becomes larger accordingly, and a large amount of user data is stored in a data management system. For the individual user, a part of the data is not required to be stored, and thus the data can be deleted, but when some user data is deleted by mistake or is required for a second time, there is a need to restore the data.
At present, a method for deleting user data is to add a special field in a database table to indicate that data is invalid when a user submits a data deletion request, and under the condition, the data still exists in the database table but cannot be displayed at a user end; when the data recovery is needed, the data recovery can be realized only by deleting the added special fields correspondingly; when the user confirms that the data is completely deleted, the data content stored in the database table can be directly deleted, and the operation is irreversible.
However, when the data deletion method is used for deleting data, under the condition that the amount of data to be deleted is large, adding the special field takes a long time, the efficiency is reduced, the operation performance of the server may be affected, and the use experience of a user is reduced.
Disclosure of Invention
The embodiment of the disclosure at least provides a data processing method, a device computer device and a computer storage medium.
In a first aspect, an embodiment of the present disclosure provides a data processing method, including: acquiring a first data processing request for adjusting the data state of user data sent by a user side; acquiring a task state of a first task corresponding to the first data processing request at the current moment to obtain a first task state, wherein the first task is a task for processing user data, and the task state of the first task is used for indicating a data state corresponding to the user data; under the condition that the user data meet preset conditions based on the first task state, the task state of the first task is adjusted to be a second task state, and the second task state is stored and sent to the user side; and determining a data processing type corresponding to the second task state, and executing the first task based on the data processing type to adjust the data state of the user data.
In a possible implementation manner, the obtaining a task state of a first task corresponding to the first data processing request at the current time to obtain a first task state includes: acquiring the data state of the user data in a storage system of a state machine based on the first data processing request; the state machine is used for recording the task state of the first task; determining a first task state of the first task based on the obtained data state of the user data.
In one possible embodiment, the executing the first task based on the data processing type includes: determining user data matching the first task; dividing the first task into at least one subtask based on the data type of the matched user data, wherein each subtask corresponds to user data of one data type; adding the at least one subtask in a task execution queue; and sequentially pulling each subtask from the task execution queue, and executing each pulled subtask based on the data processing type.
In a possible implementation manner, the pulling each subtask in turn from the task execution queue, and executing each pulled subtask based on the data processing type includes: pulling the task execution state of each subtask from the task execution queue; and executing the pulled subtasks based on the data processing type under the condition that the subtasks meet the task execution condition based on the task execution state.
In one possible embodiment, the method further comprises: in the process of executing the first task, under the condition of acquiring a second data processing request sent by the user side, determining a task state of a second task corresponding to the second data processing request to obtain a third task state; wherein the third task state is used to indicate a data state requested to be adjusted by the second data processing request; and when the third task state is different from the second task state, adjusting the processing process of the user data processed by the first task based on the third task state.
In a possible implementation manner, the adjusting the processing procedure of the user data processed by the first task based on the third task state includes: stopping the data processing process of the user data corresponding to the currently executed subtask; acquiring execution result information of the first task and the second task; and adjusting the processing process of the user data processed by the first task based on the execution result information and the second task.
In one possible embodiment, the method further comprises: and for any subtask executed at the current moment, stopping executing the subtask when determining that an exception exists in the process of executing the subtask, and moving the subtask to a specified position for storing the subtask with the exception in the task execution queue.
In one possible embodiment, the data processing type includes any one of: a data logout operation, a data recovery operation, and a data complete logout operation.
In a possible implementation manner, in a case that the data processing type is a data complete logout operation, or in a case that the data processing type is a data logout operation and a first task state of the first task is logged out, the executing the first task based on the data processing type to adjust a data state of the user data includes: and storing the user data processed by the first task to an encrypted data center, and deleting the user data processed by the first task so as to adjust the data state of the user data to be completely logged off.
In a second aspect, an embodiment of the present disclosure further provides a data processing apparatus, including: a first obtaining unit, configured to obtain a first data processing request for adjusting a data state of user data sent by a user side; a second obtaining unit, configured to obtain a first task state of a first task corresponding to the first data processing request at a current time, where the first task is a task for processing user data, and the task state of the first task is used to indicate a data state of the user data; the adjusting unit is used for adjusting the task state of the first task to a second task state under the condition that the user data meets the preset conditions based on the first task state, storing the second task state and sending the second task state to the user side; and the determining unit is used for determining the data processing type corresponding to the second task state and executing the first task based on the data processing type so as to adjust the data state of the user data.
In a third aspect, this disclosure also provides a computer device, a processor, and a memory, where the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the machine-readable instructions are executed by the processor to perform the steps in the first aspect or any one of the possible implementations of the first aspect.
In a fourth aspect, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, where the computer program is executed to perform the steps in the first aspect or any one of the possible implementation manners of the first aspect.
For the description of the effects of the data processing apparatus, the computer device and the computer storage medium, reference is made to the description of the data processing method, and details are not repeated here.
The embodiment of the disclosure provides a data processing method, a data processing device, computer equipment and a computer storage medium. In the embodiment of the disclosure, a first data processing request sent by a user side is first obtained, and then a first task state of a first task corresponding to the first data processing request at the current moment is obtained; then, under the condition that the user data meets the preset conditions based on the first task state, the task state of the first task is adjusted to be a second task state, and meanwhile, the second task state can be stored and sent to the user side; next, a data processing type corresponding to the second task state may be determined, and the first task may be executed based on the data processing type to adjust the data state of the user data.
In the embodiment of the disclosure, the first task state of the first task corresponding to the first data processing request is obtained, and then, when it is determined that the user data meets the preset condition according to the first task state, the task state of the first task is adjusted to the second task state, so that the data state of the user data is adjusted according to the second task state, the change speed of the data state of the user data can be increased, the change efficiency of the data state of the user data is increased, and the use experience of the user is further improved.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.
Fig. 1 is a schematic diagram illustrating an application scenario of a data processing method provided by an embodiment of the present disclosure;
FIG. 2 is a flow chart illustrating a data processing method provided by an embodiment of the present disclosure;
FIG. 3 illustrates a task state flow diagram provided by an embodiment of the present disclosure;
FIG. 4 illustrates a flow diagram of a data logout provided by an embodiment of the present disclosure;
FIG. 5 illustrates a flow chart of data recovery provided by an embodiment of the present disclosure;
FIG. 6 illustrates a flow chart of data deletion recovery provided by an embodiment of the present disclosure;
FIG. 7 illustrates a flow chart of a complete data logout provided by an embodiment of the present disclosure;
FIG. 8 is a schematic diagram of a data processing apparatus provided by an embodiment of the present disclosure;
fig. 9 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of embodiments of the present disclosure, as generally described and illustrated herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the disclosure, as claimed, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.
Furthermore, the terms "first," "second," and the like in the description and in the claims, and in the drawings described above, in the embodiments of the present disclosure are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein.
Reference herein to "a plurality or a number" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
It has been found that a user may generate a large amount of interactive data in the process of processing various services using various applications. Generally, a large amount of interaction data is stored in a database corresponding to each application to meet the subsequent search requirement of the user. Some of the generated interactive data is meaningless for the user, and therefore the user may need to delete the data. Alternatively, when the amount of stored data is too large, the smoothness and efficiency of the application may be affected, and thus a certain amount of interactive data needs to be deleted to reduce the storage pressure of the application.
Correspondingly, for some deleted data, it may need to be reproduced in some specific cases, and thus a recovery operation for the deleted data is also needed. The most common method for deleting/restoring data at present is to add/delete special fields to/from a data table stored in a database to delete or restore data. However, when the amount of data in the data table is too large, the process of adding/deleting the special fields takes a lot of time, reduces the efficiency of data processing, and may affect the operation performance of the application server.
Based on the above research, the present disclosure provides a data processing method, apparatus, computer device, and computer storage medium. In the embodiment of the disclosure, the first task state of the first task corresponding to the first data processing request is obtained, and then, when it is determined that the user data meets the preset condition according to the first task state, the task state of the first task is adjusted to the second task state, so that the data state of the user data is adjusted according to the second task state, the change speed of the data state of the user data can be increased, the change efficiency of the data state of the user data is increased, and the use experience of the user is further improved.
The above-mentioned drawbacks are the results of the inventor after practical and careful study, and therefore, the discovery process of the above-mentioned problems and the solutions proposed by the present disclosure to the above-mentioned problems should be the contribution of the inventor in the process of the present disclosure.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
Example one
To facilitate understanding of the present embodiment, an application scenario of the data processing method disclosed in the embodiment of the present disclosure is first introduced, and as shown in fig. 1, an application scenario schematic diagram of the data processing method provided in the embodiment of the present disclosure is shown. A user logs in the service server 12 or logs out the server 13 through a user end installed in the terminal device 11, where the user end may be a browser of a web page or an application client installed in a terminal device, such as a mobile phone, a tablet computer, or the like. The terminal device 11 is in communication connection with the service server 12 and the cancellation server 13 through a network, which may be a local area network, a cellular network, a wide area network, and the like. The terminal device 11 may be a portable device (e.g., a mobile phone, a tablet, a notebook Computer, etc.) or a Personal Computer (PC), the service server 12 and the cancellation server 13 may be any devices capable of providing internet services, in addition, the service server 12 and the cancellation server 13 are connected to the database 14, and can obtain required service data in the database 14, the database 14 includes an original database table 15 and a backup database table 16, generally, user data is stored in the original database table 15 in the database, and in case of performing a data cancellation operation on the data, the user data is stored in the backup database table 16 in the database, and the user cannot access the backup database table 16, and perform a complete cancellation operation on the user data, according to the embodiment of the present disclosure, the user data is stored in an encrypted data center, and delete the user data from the backup database table 16.
The service server 12 and the cancellation server 13 may both provide different internet services for the user, for example, the service server 12 and the cancellation server 13 may both provide data processing services for the user, in which case, the user submits a data processing request through a client (i.e., a user side) installed in the terminal device 11, the client determines the service server 12 or the cancellation server 13 executing a data processing task based on a data operation type carried in the received data processing request, and forwards the received data processing request to a corresponding server. Taking the server for determining to execute the data processing task as the cancellation server 13 as an example, the cancellation server 13 obtains the corresponding data from the database 14 according to the received data processing request to generate the corresponding task, and then implements the change of the data state by executing the task.
In specific implementation, the embodiment of the present disclosure can be implemented based on a micro service system architecture, which is a technology for deploying applications and services in the cloud and provides a set of basic architectures, such architecture enables independent deployment, operation, and upgrade of micro services, and moreover, the system architecture also enables the micro services and the micro services to be structurally loosely coupled, and functionally represent a unified whole. The micro-service can be realized by a Docker container, one physical machine can simultaneously operate a plurality of containers, or only one container can be operated, and each container bears one micro-service, so that a complex micro-service framework is realized.
Example two
The data processing method provided by the embodiment of the present disclosure is described below by taking an execution subject as a computer device as an example, and the data processing method can be applied to servers that provide different internet services for users, such as the service server 12 and the logout server 13.
As shown in fig. 2, a flowchart of a data processing method provided in an embodiment of the present disclosure may include the following steps:
s201: the method comprises the steps of obtaining a first data processing request which is sent by a user side and used for adjusting the data state of user data.
In this step, when the user has a data processing requirement, a first data processing request may be submitted at the user side based on the user data to be processed. Here, the first data processing request is a processing request for requesting to adjust the data state of the user data, and therefore, the first data processing request carries a data state identifier, which is used to indicate the user data that the user wishes to adjust and the data state of the user data that the user wishes to adjust. Wherein, the data state may comprise the following states: a data logout state, a data recovery state, and a data complete logout state.
When a user submits a first data processing request at a user side, the user side can acquire a data state identifier carried in the first data processing request, and further determine user data which the user desires to adjust and a corresponding data state according to the data state identifier.
S202: and acquiring a task state of a first task corresponding to the first data processing request at the current moment to obtain a first task state, wherein the first task is a task for processing user data, and the task state of the first task is used for indicating a data state corresponding to the user data.
In this embodiment of the present disclosure, after the first data processing request is obtained, the first task may be generated based on the first data processing request, where a first task state of the first task is a data state of user data requested to be processed by the first data processing request at the current time.
To improve the accuracy of the data processing operation, a state machine is introduced in the embodiments of the present disclosure, which is used to modify and/or record the task state, for example, modify and/or record the task state of the first task. As shown in fig. 3, a task state flow diagram provided for the embodiment of the present disclosure, where task states may include an initial state, a logged-off state, a recovered state, a logged-off state, and a completely logged-off state, and the determination of different task states is determined according to a specific data processing request submitted by a user.
In one embodiment, the task state may be stored in a remote storage space by using a KV (Key-Value) storage manner, wherein the remote storage space may be implemented by using a remote service dictionary (Redis).
S203: and under the condition that the user data meets the preset conditions based on the first task state, adjusting the task state of the first task to be a second task state, and storing and sending the second task state to the user side.
In this embodiment of the present disclosure, after determining the first task state of the first task, it may be determined whether the user data requested to be processed by the first data processing request satisfies a preset condition based on the first task state, and the method specifically includes the following steps:
determining the data state of the user data requested to be processed by the first data processing request at the current moment based on the first task state to obtain a first data state, and determining the data state of the user data requested to be adjusted by the user based on the user data requested to be processed by the first data processing request to obtain a second data state; whether the first data state and the second data state meet the state adjustment condition (i.e., the preset condition) is judged based on the state machine. And under the condition that the first task state is judged to be satisfied, the task state of the first task is adjusted from the first task state to a second task state, wherein the second task state is used for indicating the second data state.
Taking the data state that the user wants to adjust as the data logout state, after receiving the first data processing request, the logout server determines the corresponding first task according to the first data processing request, and determines the first task state corresponding to the first task. Then, whether the user data meets the preset condition can be judged based on the first task state and the data state which is desired to be adjusted by the user. And under the condition that the preset condition is met, the task state of the first task is adjusted from the first task state to the second task state, namely to a data logout state through the state machine. After adjusting the task state of the first task to a data logged off state, the second task state (e.g., data logged off state) may also be stored in the remote storage space.
In specific implementation, the cancellation server may send a state adjustment request to the state machine after acquiring the first data processing request, and the state machine invokes a state storage process according to the received state adjustment request, so as to acquire a current task state (i.e., a first task state) of the first task from the Redis according to the state storage process. When the cancellation server determines that the user data meets the preset condition according to the current task state, the state machine adjusts the task state corresponding to the first task, and specifically, the task state of the first task can be adjusted from the first task state to the second task state.
When the current task state is the "initial state", the "recovered state", or the "recovered state", the task state of the first task may be adjusted to the "logged-off state. In a case where the current task state is a "logged-off" state or a "logged-off state", the task state of the first task may be adjusted to a "recovery" state, and accordingly, a data recovery operation may be performed on the user data to be processed. When the current task state is the "logged-off state", the task state of the first task may be adjusted to a "complete logged-off" state, and accordingly, a complete logged-off operation of data of the user to be processed may be performed. After the task state adjustment of the first task is successful, the state machine may send the new task state (e.g., the second task state) to the Redis for saving.
After the task state of the first task is adjusted from the first task state to the second task state, the second task state may be stored in the remote storage space, and meanwhile, a state adjustment result may be returned to the user side. By the processing mode, response delay of the data processing operation request can be reduced, long-time waiting of a user is avoided, and user experience is improved.
S204: and determining a data processing type corresponding to the second task state, and executing the first task based on the data processing type to adjust the data state of the user data.
In the embodiment of the present disclosure, after the task state of the first task is adjusted from the first task state to the second task state, the data processing type corresponding to the second task state may also be determined. Here, the data processing type includes any one of: a data logout operation, a data recovery operation, and a data complete logout operation. The first task may then be executed based on the data processing type to adjust the data state of the user data.
In the embodiment of the disclosure, the first task state of the first task corresponding to the first data processing request is obtained, and then, when it is determined that the user data meets the preset condition according to the first task state, the task state of the first task is adjusted to the second task state, so that the data state of the user data is adjusted according to the second task state, the change speed of the data state of the user data can be increased, the change efficiency of the data state of the user data is increased, and the use experience of the user is further improved.
In an alternative embodiment, step S202: obtaining a task state of a first task corresponding to the first data processing request at the current moment, and obtaining the first task state, specifically including the following steps:
s2021: acquiring the data state of the user data in a storage system of a state machine based on a first data processing request; the state machine is used for recording the task state of the first task;
s2022: determining a first task state of the first task based on the obtained data state of the user data.
As can be seen from the above description, the state machine is used to record (e.g., modify and/or store) the task state of the first task; in addition to this, the state machine can also be used to record the data state of the user data.
In the embodiment of the present disclosure, after the first data processing request is obtained, the user data requested to be processed by the user may be determined, and the current data state of the user data requested to be processed by the user is determined based on the state machine; and using the determined current data state as the current task state of the first task (i.e., the first task state).
In the embodiment of the disclosure, when the state of the user data is adjusted, by introducing the task system and the state machine, the mutual influence between different data processing flows (for example, a logout flow and a recovery flow) can be avoided, so that the data change speed is increased.
In an alternative embodiment, S204: executing the first task based on the data processing type, comprising the steps of:
s2041: determining user data matching the first task;
s2042: dividing the first task into at least one subtask based on the data type of the matched user data, wherein each subtask corresponds to user data of one data type;
s2043: adding the at least one subtask in a task execution queue;
s2044: and sequentially pulling each subtask from the task execution queue, and executing each pulled subtask based on the data processing type.
In the embodiment of the disclosure, the cancellation server may determine, when executing the first task, the user data requested to be adjusted by the user as the user data matched with the first task. Then, the data size of the user data is determined, and when the data size is determined to be large (for example, larger than a preset number threshold), if the first task is directly executed, the performance of the server may be affected, so that the first task may be split into a plurality of independent sub-tasks.
In specific implementation, the first task may be divided into at least one subtask based on a data type of the user data matched with the first task, where each subtask corresponds to a data type of the user data. After the server splits the first task into several independent subtasks, each subtask may be sent to a message queue to generate a task execution queue.
During specific implementation, the server sequentially pulls the subtasks stored in the message queue by using the called task processing process and acquires the detail information of the subtasks, and executes each pulled subtask according to the acquired detail information until all subtasks are executed, which indicates that the data processing task is completed. The process of executing the subtask is a process of deleting or recovering the corresponding user data according to the first data processing request submitted by the user, for example, the deleted or recovered user data may be browsing history, praise record, collection record, relationship chain, and the like of the user.
Further, if the first task executed by the cancellation server is a data cancellation task, the cancellation server may obtain user data matched with the data cancellation task from an original data table in a database storing corresponding data, and delete the user data. It should be noted that, while deleting the corresponding user data, the logout server may also store the deleted user data in a backup data table in the database, but in the user side, these backup data are not accessible to the user. In addition, for all subtasks included in executing the first task, if all subtasks are completely executed, the cancellation server is notified that the first task is completely executed, and the task state of the first task has been adjusted to the cancelled state.
If the cancellation server executes the data recovery task, the cancellation server may obtain the user data matched with the data recovery task from the backup data table in the database storing the corresponding data, and perform the recovery operation on the user data. In the process of recovering the user data, the cancellation server can return a message that the data is being recovered to the user side until all the subtasks are executed, and the user can obtain the corresponding data at the user side.
In an alternative embodiment, S2044: sequentially pulling each subtask from the task execution queue, and executing each pulled subtask based on the data processing type, which specifically includes the following steps:
(1) pulling the task execution state of each subtask from the task execution queue;
(2) and executing the pulled subtasks based on the data processing type under the condition that the subtasks meet the task execution condition based on the task execution state.
In the embodiment of the present disclosure, when pulling each subtask from the task execution queue, the cancellation server may obtain a task execution state of the subtask, for example, the task execution state may be the following state: execution exception, execution complete, executing, not executing. After the task execution state is acquired, whether the subtask satisfies the task execution condition can be determined based on the task execution state.
In specific implementation, if the task execution state is any one of abnormal execution, completion of execution and execution, the task execution condition is determined not to be satisfied. And if the task execution state is not executed, determining that the task execution condition is met. In the case where it is determined that the task execution condition is satisfied, the pulled subtask can be executed based on the data processing type.
By the processing mode, the first task is divided into the plurality of subtasks, and the plurality of subtasks are executed asynchronously, so that the execution parallelism of the subtasks can be improved, and the data change speed can be improved.
In an embodiment of the present disclosure, the method further includes:
and for any subtask executed at the current moment, stopping executing the subtask when determining that an exception exists in the process of executing the subtask, and moving the subtask to a specified position for storing the subtask with the exception in the task execution queue.
In the embodiment of the present disclosure, in the process of executing each pulled subtask, if an execution result of a certain subtask is abnormal, and if the task is executed all the time, which may cause other subtasks to be unable to complete normally, the task processing process may resend the subtask whose execution result is abnormal to a specified position of the task execution queue, for example, the end of the task execution queue, so as to continue executing other subtasks, and after the execution of other subtasks is completed, the task that is executed abnormally is processed, thereby ensuring the efficiency and flexibility of data processing.
In an optional embodiment, in a case that the data processing type is a data complete logout operation, or in a case that the data processing type is a data logout operation and the first task state of the first task is logged out, S204: executing the first task based on the data processing type to adjust the data state of the user data, specifically comprising the following steps:
and storing the user data processed by the first task to an encrypted data center, and deleting the user data processed by the first task so as to adjust the data state of the user data to be completely logged off.
In specific implementation, when the data processing type is data complete logout operation and the current task state of the first task is logged out, after the subtask is pulled in the task queue to obtain the subtask detail information, the user data processed by the subtask can be stored in the encrypted data center, and the information in the corresponding backup data table is deleted.
In some specific scenarios, for data stored in the encrypted data center, an approval process may be initiated, and a specific key may be available after the approval process is completed, and the specific data stored in the encrypted data center may be accessed through the specific key. In the embodiment of the present disclosure, the user data stored in the encrypted data center may be periodically deleted according to a certain preset period, and in general, the default period is two years, and the data content deleted from the encrypted data center is completely deleted and cannot be acquired.
In an alternative embodiment, the method further comprises the steps of:
(1) in the process of executing the first task, under the condition of acquiring a second data processing request sent by the user side, determining a task state of a second task corresponding to the second data processing request to obtain a third task state; wherein the third task state is used to indicate a data state requested to be adjusted by the second data processing request;
(2) and when the third task state is different from the second task state, adjusting a processing process of the user data processed by the first task based on the third task state.
During specific implementation, the server can also receive a second data processing request submitted by a user at a user side in the process of calling a task processing process to execute a subtask; the second data processing request includes user data requested to be processed by the user and a data state of the user data requested to be processed.
It is assumed that the first data processing request is a data logout request and the second data processing request is a data recovery request. At this time, the task state of the second task corresponding to the second data processing request may be determined, that is, the third task state is: and recovering the state of the data. The embodiment of the disclosure allows a user to recover deleted data in the process of logging off the data. Based on this, the cancellation server determines the second task according to the received second data processing request, and determines the task state of the second task to obtain a third task state. In the embodiment of the present disclosure, the user data that needs to be processed by the first task is the same user data as the user data that needs to be processed by the second task.
After the third task state is determined, the third task state may be compared with the second task state, and when the third task state is different from the second task state, a processing procedure of the user data processed by the first task is adjusted based on the third task state, where the specific processing procedure is described as follows:
acquiring a task processing result of a first task at the current moment, wherein the task processing result of the first task is used for indicating operated data and non-operated data in corresponding user data; and adjusting the processing procedure of the user data processed by the first task based on the task processing result.
For the non-operation data, it may be determined whether data processing is required for the non-operation data based on the third task state. And if the data is needed, executing a second task to adjust the data state of the data which is not operated. If it is determined that no operation is required, no operation may be performed on the non-operated data.
For example, the first data processing request is a data logout request, the second data processing request is a data recovery request, and for data which is not operated, the data is still stored in a source data table of the database, so that the recovery operation is not required to be performed on the data.
For the operated data, a second task needs to be executed to adjust the data state of the operated data. For example, the first data processing request is a data logout request, the second data processing request is a data recovery request, and for the operated data, since the operated data is backed up in a backup data table of the database, a data recovery operation is required to be performed to obtain the operated data in the main data table. Thus, the cancellation server may perform the second task, and then perform a data recovery operation on the operated data by performing the second task.
As can be seen from the above description, different tasks are determined for different data processing requests, and by executing each task separately, the interaction between the tasks can be avoided, thereby increasing the change speed of the data state.
For better understanding of the embodiments of the present disclosure, a specific implementation process of the embodiments of the present disclosure is described below with reference to a data logout operation process, a data recovery operation process, and a process of performing a data recovery operation during a data logout process. Based on the micro-service architecture system, in the embodiment of the present disclosure, the state machine function, the KV storage function, the message queue function, and the task processing function are all deployed on the server in the form of micro-service, and in the specific implementation, they may be deployed on the cancellation server, or may be deployed on different servers, which is not limited in the embodiment of the present disclosure. The cancellation server may implement the call of the corresponding function by initiating a network request to the server deploying the corresponding micro service, for example, the cancellation server may implement the call of the task processing process by sending a network request to the server deploying the task processing function.
EXAMPLE III
Fig. 4 is a flowchart of data logout provided by an embodiment of the present disclosure, where a consumer is a micro service consuming a task, a concept corresponding to the consumer is a producer, the producer and the consumer are abstract concepts, the producer is the micro service submitting the task, the producer is responsible for producing data and storing the data in a message queue, and the consumer is responsible for fetching the data from the queue for consumption, which are communicated through the message queue. In the embodiment of the disclosure, the consumer is a micro service for realizing the task processing function. Based on this, as shown in fig. 4, the data logout flow may include the following steps:
s401: and receiving a data logout request, namely the first data processing request.
In specific implementation, a user sends a data logout request to a logout server through a user side under the condition that the user needs to logout data, wherein the data logout request carries a data identifier of user data to be logout.
Further, the cancellation server determines a data cancellation task, namely a first task, according to the data identifier.
S402: the cancellation server sends a task state adjustment request to the state machine.
In specific implementation, in order to improve accuracy of data processing and data processing efficiency, a state machine is introduced to manage and control task states in the embodiment of the disclosure. Based on this, before the cancellation server calls the relevant micro-service to perform task processing, it needs to judge whether the task state is a preset state.
Based on this, the cancellation server sends a task state adjustment request to the state machine, in this example, the cancellation server initiates a request to the state machine to adjust the task state to "cancellation".
S403: and pulling the current task state, namely the first task state, from the KV memory system by the state machine.
Taking the example that the KV storage is implemented by the Redis, in this step, the state machine acquires the current state of the data logout task, i.e., the first task state, from the Redis according to the data identifier of the user data to be logout.
As can be seen from fig. 3, the default state of the data logout task is set as the initial state, and the task state of the first task is subsequently adjusted according to the user request. In the initial state, the task state can only be adjusted to be a logout state, that is, only a user is allowed to logout the data.
S404: and returning the task state to the state machine by the KV memory system.
In this step, the KV storage system queries a corresponding task state (i.e., a first task state) according to the data identifier of the user data to be logged out, and sends the queried task state to the state machine.
S405: the state machine adjusts the task state.
In this step, the cancellation server adjusts the current task state of the data cancellation task to the data cancellation state when determining that the user data to be cancelled meets a preset condition (e.g., cancellation condition) according to the current task state and the data cancellation request.
As shown in fig. 3, for the data logout task, only in the case where the task state is the initial state, the recovery state, or the recovered state, the task state can be adjusted to the data logout state. If the task state is the other task state, the task state cannot be adjusted, namely the user is not allowed to perform data logout operation, and in this case, a response message of data logout failure is returned to the user through the logout server.
S406: and the state machine sends the adjusted task state to the KV storage system for storage.
In this step, the state machine adjusts the task state to be a logout state, and then sends the task state to the KV storage system for storage.
S407: and returning the storage result to the state machine by the KV storage system.
S408: and the state machine sends a task state adjustment result to the cancellation server.
S409: the cancellation server deposits the generated data cancellation task into a message queue (i.e., the task execution queue).
S410: the message queue returns the stored results to the cancellation server.
S411: the cancellation server sends the cancellation result to the client (or the user side).
In this step, after receiving a response message that the data logout task returned by the message queue is successfully stored, the logout server returns a response message that the data logout is successfully logged back to the user through the client.
S412: the consumer pulls a data logout task from the message queue.
S413: the message queue returns data cancellation task detail information to the consumer.
Wherein, the details information of the data logout task comprises the user data to be logout.
S414: the consumer splits the data log-out task.
In the step, the consumer divides the data cancellation task into a plurality of data annotation task according to the received details of the data cancellation task.
S415: and the consumer sends the split data logout subtask to a message queue for storage.
In the step, the consumer sends the split data annotation task to the message queue for storage.
S416: the message queue returns the stored results to the consumer.
S417: the consumer pulls data in turn from the message queue to log off the subtasks.
S418: the message queue returns data cancellation subtask details to the consumer.
S419: the consumer performs a data logout subtask.
S420: the consumer sends the currently executing data job to the message queue.
In this step, in the process of executing the data cancellation subtask by the consumer, if the execution result of the data annotation subtask is abnormal, the data cancellation subtask is sent to the message queue and stored at the tail end of the message queue.
S421: the message queue returns the stored results to the consumer.
In this step, after the consumer executes other data annotating task except the data logout subtask with abnormal execution result, the consumer continues to pull the abnormal data annotating task from the message queue for execution until all the data logout subtasks are completed, and returns a response message of successful data logout to the logout server, and further, the logout server calls a state machine to adjust the task state to be the logout state, and the process is ended.
Example four
As shown in fig. 5, which is a flowchart of data recovery provided by an embodiment of the present disclosure, the method may include the following steps:
s501: and receiving a data recovery request, namely the first data processing request.
In specific implementation, when a user needs to recover the logged-off data, a data recovery request is sent to the logging-off server through the user side, and the data recovery request carries the data identifier of the user data to be recovered.
Further, the cancellation server determines to generate a data recovery task, i.e. a first task, according to the data identifier of the data to be recovered.
S502: the cancellation server sends a task state adjustment request to the state machine.
In specific implementation, the cancellation server sends a task state adjustment request to the state machine, and in this example, the cancellation server initiates a request for adjusting the task state to "resume" to the state machine.
S503: and pulling the current task state, namely the first task state, from the KV memory system by the state machine.
Taking the example that the KV storage is implemented by the Redis, in this step, the state machine acquires the current state of the data recovery task, that is, the first task state, from the Redis according to the data identifier of the user data to be recovered.
S504: and returning the task state to the state machine by the KV memory system.
In this step, the KV storage system queries a corresponding task state (i.e., a first task state) according to the data identifier of the user data to be restored, and sends the queried task state to the state machine.
S505: the state machine adjusts the task state.
In this step, the current task state of the data cancellation task is adjusted to the data cancellation state when the cancellation server determines that the user data to be restored meets a preset condition (e.g., a cancellation condition) according to the current task state and the data restoration request.
As shown in fig. 3, for the data recovery task, only in the case where the task state is the logged-off state or the logged-off state, the task state can be adjusted to the data recovery state. If the task state is the other task state, the task state cannot be adjusted, namely the user is not allowed to perform the data recovery operation, and in this case, a response message of failure in data recovery is returned to the user through the cancellation server.
S506: and the state machine sends the adjusted latest task state to the KV storage system for storage.
In this step, the state machine adjusts the task state to be a logout state, and then sends the task state to the KV storage system for storage.
S507: and returning the storage result to the state machine by the KV storage system.
S508: and the state machine sends a task state adjustment result to the cancellation server.
S509: the cancellation server stores the generated data recovery task in a message queue (i.e., the task execution queue described above).
S510: the message queue returns the stored results to the cancellation server.
S511: and the cancellation server sends the recovery result to the client.
In this step, after receiving the response message that the data recovery task returned by the message queue is successfully stored, the cancellation server returns the response message that the data recovery is successful to the user through the client.
S512: the consumer pulls the data recovery task from the message queue.
S513: the message queue returns data recovery task detail information to the consumer.
The data recovery task detail information comprises user data to be recovered.
S514: the consumer splits the data recovery task.
In this step, the consumer splits the data recovery task into a plurality of data recovery subtasks according to the received details of the data recovery task.
S515: and the consumer sends the split data recovery subtasks to the message queue for storage.
In the step, the consumer sends the split data annotation task to the message queue for storage.
S516: the message queue returns the stored results to the consumer.
S517: the consumer pulls data in turn from the message queue to recover the subtasks.
S518: the message queue returns data recovery subtask details to the consumer.
S519: the consumer performs a data recovery subtask.
S520: the consumer sends the currently executing data recovery subtask to the message queue.
In this step, in the process of executing the data recovery subtask by the consumer, if the execution result of the data recovery subtask is abnormal, the data recovery subtask is sent to the message queue and stored at the tail end of the message queue.
S521: the message queue returns the stored results to the consumer.
In this step, after the consumer executes the other data recovery subtasks except the data recovery subtask whose execution result is abnormal, the consumer continues to pull the abnormal data recovery subtask from the message queue to execute until all the data recovery subtasks are completed, and returns a response message indicating that the data recovery is successful to the cancellation server, and further, the cancellation server calls a state machine to adjust the task state to the recovered state, and the process is ended.
EXAMPLE five
As shown in fig. 6, which is a flowchart for recovering data deletion provided by the embodiment of the present disclosure, the method may include the following steps:
s601: and receiving a data logout request, namely the first data processing request.
In specific implementation, a user sends a data logout request to a logout server through a user side under the condition that the user needs to logout data, wherein the data logout request carries a data identifier of user data to be logout.
S602: the cancellation server sends a task state adjustment request to the state machine.
In this step, the cancellation server sends a task state adjustment request to the state machine, and in this example, the cancellation server initiates a request for adjusting the task state to be "cancelled" to the state machine.
S603: and pulling the current task state, namely the first task state, from the KV memory system by the state machine.
Taking the example that the KV storage is implemented by the Redis, in this step, the state machine acquires the current state of the data logout task, i.e., the first task state, from the Redis according to the data identifier of the user data to be logout.
S604: and returning the task state to the state machine by the KV memory system.
In this step, the KV storage system queries a corresponding task state (i.e., a first task state) according to the data identifier of the user data to be logged out, and sends the queried task state to the state machine.
S605: the state machine adjusts the task state.
In this step, the cancellation server adjusts the current task state of the data cancellation task to the data cancellation state when determining that the user data to be cancelled meets a preset condition (e.g., cancellation condition) according to the current task state and the data cancellation request.
As shown in fig. 3, for the data logout task, only in the case where the task state is the initial state, the recovery state, or the recovered state, the task state can be adjusted to the data logout state. If the task state is the other task state, the task state cannot be adjusted, namely the user is not allowed to perform data logout operation, and in this case, a response message of data logout failure is returned to the user through the logout server.
S606: and the state machine sends the adjusted task state to the KV storage system for storage.
In this step, the state machine adjusts the task state to be a logout state, and then sends the task state to the KV storage system for storage.
S607: and returning the storage result to the state machine by the KV storage system.
S608: and the state machine sends a task state adjustment result to the cancellation server.
S609: the cancellation server deposits the generated data cancellation task into a message queue (i.e., the task execution queue).
S610: the message queue returns the stored results to the cancellation server.
S611: and the logout server sends the logout result to the client.
In this example, after receiving a response message that the data logout task returned by the message queue is successfully stored, the logout server returns a response message that the data logout is successfully logged out to the user through the client.
S612: the consumer pulls a data logout task from the message queue.
S613: the message queue returns data cancellation task detail information to the consumer.
Wherein, the details information of the data logout task comprises data to be logout.
S614: the consumer splits the data log-out task.
In the step, the consumer divides the data cancellation task into a plurality of data annotation task according to the received details of the data cancellation task.
S615: and the consumer sends the split data logout subtask to a message queue for storage.
In the step, the consumer sends the split data annotation task to the message queue for storage.
S616: the message queue returns the stored results to the consumer.
S617: the consumer pulls data in turn from the message queue to log off the subtasks.
S618: the message queue returns data cancellation subtask details to the consumer.
S619: the consumer performs a data logout subtask.
S620: the consumer sends the currently executing data job to the message queue.
In this step, in the process of executing the data cancellation subtask by the consumer, if the execution result of the data annotation subtask is abnormal, the data cancellation subtask is sent to the message queue and stored at the tail end of the message queue.
S621: a data recovery request, i.e. the second data processing request described above, is received.
In this step, the cancellation server may also invoke the state machine to read the task state from the KV storage system to determine whether a new data processing request (i.e., the second data processing request) is received in the process of invoking the task processing process program to execute the data annotation task. In a case where it is determined that the second data processing request is received, execution of the data annotating task is stopped.
In specific implementation, in the process of executing a data annotation task corresponding to a data logout task, when a user needs to recover logged-out data, a data recovery request is sent to a logout server through a client, wherein the data recovery request carries a data identifier of user data to be recovered. Further, the cancellation server receives a data recovery request from the client, and after generating a data recovery task (a second task) according to the data identifier of the user data to be recovered, the cancellation server stops invoking the task processing process program, stops processing the unexecuted data cancellation subtask, and continues executing the data recovery task.
S622: the cancellation server sends a task state adjustment request to the state machine.
In specific implementation, the cancellation server sends a task state adjustment request to the state machine, and in this example, the cancellation server initiates a request for adjusting the task state to "resume" to the state machine.
S623: and pulling the current task state from the KV memory system by the state machine.
Taking the example that the KV storage is implemented by the Redis, in this step, the state machine acquires the current task state of the data recovery task from the Redis according to the data identifier of the user data to be recovered.
S624: and returning the task state to the state machine by the KV memory system.
In this step, the KV storage system queries a corresponding task state according to the data identifier of the user data to be restored, and sends the queried task state to the state machine.
S625: the state machine adjusts the task state.
In this step, the cancellation server adjusts the current task state of the data recovery task to the data recovery state when determining that the user data to be recovered satisfies a preset condition (e.g., a recovery condition) according to the current task state and the data recovery request.
As shown in fig. 3, for the data recovery task, only in the case where the task state is the logged-off state or the logged-off state, the task state can be adjusted to the data recovery state. If the task state is the other task state, the task state cannot be adjusted, namely the user is not allowed to perform the data recovery operation, and in this case, a response message of failure in data recovery is returned to the user through the cancellation server.
S626: and the state machine sends the adjusted latest task state to the KV storage system for storage.
In this step, the state machine adjusts the task state to the data recovery state, and then sends the data recovery state to the KV storage system for storage.
S627: and returning the storage result to the state machine by the KV storage system.
S628: and the state machine sends a task state adjustment result to the cancellation server.
S629: and the cancellation server stores the generated data recovery task in a message queue.
S630: the message queue returns the stored results to the cancellation server.
S631: and the cancellation server sends the recovery result to the client.
In this example, after receiving a response message that the data recovery task returned by the message queue is successfully stored, the cancellation server returns a response message that the data recovery is successful to the user through the client.
S632: the consumer pulls the data recovery task from the message queue.
S633: the message queue returns data recovery task detail information to the consumer.
The data recovery task detail information comprises data to be recovered.
S634: the consumer splits the data recovery task.
In this step, the consumer splits the data recovery task into a plurality of data recovery subtasks according to the received details of the data recovery task.
S635: and the consumer sends the split data recovery subtasks to the message queue for storage.
In this step, the consumer generates a task processing queue according to the split data recovery subtasks and sends the task processing queue to a message queue for storage.
S636: the message queue returns the stored results to the consumer.
S637: the consumer pulls data in turn from the message queue to recover the subtasks.
S638: the message queue returns data recovery subtask details to the consumer.
S639: the consumer performs a data recovery subtask.
S640: the consumer sends the currently executing data recovery subtask to the message queue.
In this step, in the process of executing the data recovery subtask by the consumer, if the execution result of the data recovery subtask is abnormal, the data recovery subtask is returned to the message queue and stored at the tail end of the message queue.
S641: the message queue returns the stored results to the consumer.
In this step, after the consumer executes the other data recovery subtasks except the data recovery subtask whose execution result is abnormal, the consumer continues to pull the abnormal data recovery subtask from the message queue to execute until all the data recovery subtasks are completed, and returns a response message indicating that the data recovery is successful to the cancellation server, and further, the cancellation server calls a state machine to adjust the task state to the recovered state, and the process is ended.
EXAMPLE six
As shown in fig. 7, which is a flowchart of a complete data logout provided by an embodiment of the present disclosure, the method may include the following steps:
s701: and receiving a data complete logout request, namely the first data processing request.
In specific implementation, a user sends a data complete logout request to a logout server through a client under the condition that the user needs to completely logout data, wherein the data complete logout request carries a data identifier of user data to be completely logout.
Further, the cancellation server determines a data complete cancellation task, i.e. a first task, according to the data identifier of the completely cancelled user data.
S702: the cancellation server sends a task state adjustment request to the state machine.
In this step, the cancellation server sends a task state adjustment request to the state machine, and in this example, the cancellation server initiates a request for adjusting the task state to be "completely cancelled" to the state machine.
S703: and pulling the current task state, namely the first task state, from the KV memory system by the state machine.
Taking the example that the KV storage is realized by Redis, in this step, the state machine acquires the current task state of the task in which the data is completely logged out from the Redis according to the data identifier of the user data completely logged out.
S704: and returning the current task state to the state machine by the KV memory system.
In this step, the KV storage system queries the corresponding task state according to the data identifier of the user data completely logged out, and sends the queried task state to the state machine.
S705: the state machine adjusts the task state.
In this step, when the cancellation server determines that the user data to be completely cancelled meets a preset condition (for example, a cancellation condition) according to the current task state and the complete data cancellation request, the current task state of the complete data cancellation task is adjusted to the complete data cancellation state.
As shown in fig. 3, for a data complete logout task, only in the case where the task state is logged out, the task state can be adjusted to a complete logout state. If the task state is other task state, the task state cannot be adjusted, that is, the user is not allowed to perform data logout operation, in this case, a response message of completely logging out the data failure is returned to the user through the logout server.
S706: and the state machine sends the adjusted task state to the KV storage system for storage.
In this example, the state machine adjusts the task state to a state in which data is completely logged off, and then sends the task state to the KV storage system for storage.
S707: and returning the storage result to the state machine by the KV storage system.
S708: and the state machine sends a task state adjustment result to the cancellation server.
S709: and the cancellation server stores the generated data complete cancellation task in a message queue.
S710: the message queue returns the stored results to the cancellation server.
S711: and the logout server sends a complete logout result to the client.
In this example, after receiving the response message that the data complete logout task returned by the message queue is successfully stored, the logout server returns the response message that the data complete logout is successfully logged back to the user through the client.
S712: the consumer pulls data from the message queue to completely log off the task.
S713: the message queue returns data complete logout task detail information to the consumer.
Wherein, the details information of the data logout task comprises data to be logout.
S714: the consumer splits the data and completely logs out the task.
In the step, the consumer splits the data complete logout task into a plurality of data complete logout tasks according to the received detailed information of the data complete logout task.
S715: and the consumer sends the split data complete cancellation subtask to the message queue for storage.
In the step, the consumer completely annotates the pin task according to the split data and sends the pin task to the message queue for storage.
S716: the message queue returns the stored results to the consumer.
S717: the consumer pulls data from the message queue in sequence to completely log off the subtasks.
S718: the message queue returns data complete logout subtask details to the consumer.
S719: and the consumer sends the data to a data unloading encryption data center for storage.
In this step, in order to ensure that the data completely deleted by the user can be found under some conditions that need to be verified, the consumer stores the data information received from each data complete logout subtask into the data dump encrypted data center and deletes the corresponding backup data stored in the backup data table in the database, and in addition, the data stored in the data dump encrypted data center is invisible to the user's incoming book, and the data stored in the data dump encrypted data center can be checked only under the condition that a specific key is obtained.
S720: and the data is transferred to the encrypted data center, and the storage result is returned to the consumer.
S721: the consumer performs a complete logoff subtask.
S722: the consumer sends the currently executed data to the message queue to completely annotate the pin task.
In this step, in the process of executing the complete cancellation of the subtask by the consumer, if the execution result of the complete data cancellation subtask is abnormal, the complete data cancellation subtask is returned to the message queue and the tail end of the message queue is stored.
S723: the message queue returns the stored results to the consumer.
In the step, after the consumer executes other data complete pin injection tasks except the data complete pin injection subtask with abnormal execution result, the consumer continues to pull the abnormal data complete pin injection task from the message queue for execution until all data complete pin injection subtasks are completed, a response message that the data complete pin injection is successful is returned to the cancellation server, and further, the cancellation server calls a state machine to adjust the task state to a complete cancellation state, and the process is ended.
It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.
Based on the same inventive concept, a data processing apparatus corresponding to the data processing method is also provided in the embodiments of the present disclosure, and because the principle of the apparatus in the embodiments of the present disclosure for solving the problem is similar to the data processing method described above in the embodiments of the present disclosure, the implementation of the apparatus may refer to the implementation of the method, and repeated details are not described again.
EXAMPLE seven
As shown in fig. 8, a schematic diagram of a data processing apparatus provided in an embodiment of the present disclosure includes:
a first obtaining unit 81, configured to obtain a first data processing request for adjusting a data state of user data sent by a user side;
a second obtaining unit 82, configured to obtain a first task state of a first task corresponding to the first data processing request at the current time, where the first task is a task for processing user data, and the task state of the first task is used to indicate a data state of the user data;
an adjusting unit 83, configured to adjust the task state of the first task to a second task state, store the second task state, and send the second task state to the user side when it is determined that the user data meets a preset condition based on the first task state;
a determining unit 84, configured to determine a data processing type corresponding to the second task state, and execute the first task based on the data processing type to adjust the data state of the user data.
In a possible implementation, the second obtaining unit is further configured to: acquiring the data state of the user data in a storage system of a state machine based on the first data processing request; the state machine is used for recording the task state of the first task; determining a first task state of the first task based on the obtained data state of the user data.
In a possible implementation, the determining unit is further configured to: determining user data matching the first task; dividing the first task into at least one subtask based on the data type of the matched user data, wherein each subtask corresponds to user data of one data type; adding the at least one subtask in a task execution queue; and sequentially pulling each subtask from the task execution queue, and executing each pulled subtask based on the data processing type.
In a possible implementation, the determining unit is further configured to: pulling the task execution state of each subtask from the task execution queue; and executing the pulled subtasks based on the data processing type under the condition that the subtasks meet the task execution condition based on the task execution state.
In one possible embodiment, the apparatus is further configured to: in the process of executing the first task, under the condition of acquiring a second data processing request sent by the user side, determining a task state of a second task corresponding to the second data processing request to obtain a third task state; wherein the third task state is used to indicate a data state requested to be adjusted by the second data processing request; and when the third task state is different from the second task state, adjusting the processing process of the user data processed by the first task based on the third task state.
In one possible embodiment, the apparatus is further configured to: stopping the data processing process of the user data corresponding to the currently executed subtask; acquiring execution result information of the first task and the second task; and adjusting the processing process of the user data processed by the first task based on the execution result information and the second task.
In one possible embodiment, the apparatus is further configured to: and for any subtask executed at the current moment, stopping executing the subtask when determining that an exception exists in the process of executing the subtask, and moving the subtask to a specified position for storing the subtask with the exception in the task execution queue.
In one possible embodiment, the data processing type includes any one of: a data logout operation, a data recovery operation, and a data complete logout operation.
In a possible implementation, the determining unit is further configured to: and when the data processing type is a data complete logout operation, or when the data processing type is the data logout operation and the first task state of the first task is logged out, storing the user data processed by the first task to an encrypted data center, and deleting the user data processed by the first task so as to adjust the data state of the user data to be the data complete logout.
The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.
An embodiment of the present disclosure further provides a computer device, as shown in fig. 9, which is a schematic structural diagram of the computer device provided in the embodiment of the present disclosure, and the computer device includes:
a processor 91 and a memory 92; the memory 92 stores machine-readable instructions executable by the processor 91, the processor 91 being configured to execute the machine-readable instructions stored in the memory 92, the processor 91 performing the following steps when the machine-readable instructions are executed by the processor 91:
acquiring a first data processing request for adjusting the data state of user data sent by a user side; acquiring a task state of a first task corresponding to the first data processing request at the current moment to obtain a first task state, wherein the first task is a task for processing user data, and the task state of the first task is used for indicating a data state corresponding to the user data; under the condition that the user data meet preset conditions based on the first task state, the task state of the first task is adjusted to be a second task state, and the second task state is stored and sent to the user side; and determining a data processing type corresponding to the second task state, and executing the first task based on the data processing type to adjust the data state of the user data.
The memory 92 includes a memory 921 and an external memory 922; the memory 921 is also referred to as an internal memory, and temporarily stores operation data in the processor 91 and data exchanged with an external memory 922 such as a hard disk, and the processor 91 exchanges data with the external memory 922 through the memory 921.
For the specific execution process of the instruction, reference may be made to the steps of the data processing method described in the embodiments of the present disclosure, and details are not described here.
The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the data processing method described in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.
The computer program product of the data processing method provided in the embodiments of the present disclosure includes a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute steps of the data processing method in the above method embodiments, which may be referred to specifically for the above method embodiments, and are not described herein again. The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims (12)

1. A data processing method, comprising:
acquiring a first data processing request for adjusting the data state of user data sent by a user side;
acquiring a task state of a first task corresponding to the first data processing request at the current moment to obtain a first task state, wherein the first task is a task for processing user data, and the task state of the first task is used for indicating a data state corresponding to the user data;
under the condition that the user data meet preset conditions based on the first task state, the task state of the first task is adjusted to be a second task state, and the second task state is stored and sent to the user side;
and determining a data processing type corresponding to the second task state, and executing the first task based on the data processing type to adjust the data state of the user data.
2. The method according to claim 1, wherein the obtaining the task state of the first task corresponding to the first data processing request at the current time to obtain the first task state comprises:
acquiring the data state of the user data in a storage system of a state machine based on the first data processing request; the state machine is used for recording the task state of the first task;
determining a first task state of the first task based on the obtained data state of the user data.
3. The method of claim 1, wherein said executing the first task based on the data processing type comprises:
determining user data matching the first task;
dividing the first task into at least one subtask based on the data type of the matched user data, wherein each subtask corresponds to user data of one data type;
adding the at least one subtask in a task execution queue;
and sequentially pulling each subtask from the task execution queue, and executing each pulled subtask based on the data processing type.
4. The method of claim 3, wherein the pulling each subtask from the task execution queue in turn, and based on the data processing type, executing each pulled subtask comprises:
pulling the task execution state of each subtask from the task execution queue;
and executing the pulled subtasks based on the data processing type under the condition that the subtasks meet the task execution condition based on the task execution state.
5. The method of claim 1, further comprising:
in the process of executing the first task, under the condition of acquiring a second data processing request sent by the user side, determining a task state of a second task corresponding to the second data processing request to obtain a third task state; wherein the third task state is used to indicate a data state requested to be adjusted by the second data processing request;
and when the third task state is different from the second task state, adjusting the processing process of the user data processed by the first task based on the third task state.
6. The method of claim 5, wherein adjusting the processing of the user data processed by the first task based on the third task state comprises:
stopping the data processing process of the user data corresponding to the currently executed subtask;
acquiring execution result information of the first task and the second task;
and adjusting the processing process of the user data processed by the first task based on the execution result information and the second task.
7. The method of claim 3, further comprising:
and for any subtask executed at the current moment, stopping executing the subtask when determining that an exception exists in the process of executing the subtask, and moving the subtask to a specified position for storing the subtask with the exception in the task execution queue.
8. The method of claim 1, wherein the data processing type comprises any one of: a data logout operation, a data recovery operation, and a data complete logout operation.
9. The method of claim 8, wherein in a case that the data processing type is a data-through logout operation or the data processing type is a data logout operation and a first task state of the first task is logged, the executing the first task based on the data processing type to adjust the data state of the user data comprises:
and storing the user data processed by the first task to an encrypted data center, and deleting the user data processed by the first task so as to adjust the data state of the user data to be completely logged off.
10. A data processing apparatus, comprising:
a first obtaining unit, configured to obtain a first data processing request for adjusting a data state of user data sent by a user side;
a second obtaining unit, configured to obtain a first task state of a first task corresponding to the first data processing request at a current time, where the first task is a task for processing user data, and the task state of the first task is used to indicate a data state of the user data;
the adjusting unit is used for adjusting the task state of the first task to a second task state under the condition that the user data meets the preset conditions based on the first task state, storing the second task state and sending the second task state to the user side;
and the determining unit is used for determining the data processing type corresponding to the second task state and executing the first task based on the data processing type so as to adjust the data state of the user data.
11. A computer device, comprising: a processor, a memory storing machine-readable instructions executable by the processor, the processor for executing the machine-readable instructions stored in the memory, the processor performing the steps of the data processing method of any one of claims 1 to 10 when the machine-readable instructions are executed by the processor.
12. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when executed by a computer device, performs the steps of the data processing method according to any one of claims 1 to 10.
CN202110910193.4A 2021-08-09 2021-08-09 Data processing method and device, computer equipment and computer storage medium Pending CN113515516A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110910193.4A CN113515516A (en) 2021-08-09 2021-08-09 Data processing method and device, computer equipment and computer storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110910193.4A CN113515516A (en) 2021-08-09 2021-08-09 Data processing method and device, computer equipment and computer storage medium

Publications (1)

Publication Number Publication Date
CN113515516A true CN113515516A (en) 2021-10-19

Family

ID=78068022

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110910193.4A Pending CN113515516A (en) 2021-08-09 2021-08-09 Data processing method and device, computer equipment and computer storage medium

Country Status (1)

Country Link
CN (1) CN113515516A (en)

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1241882A (en) * 1998-07-02 2000-01-19 日本电气株式会社 Method of subscriber data control in mobile communication network
CN101796487A (en) * 2007-08-10 2010-08-04 内特可力亚斯系统股份有限公司 Virtual queue processing circuit and task processor
CN101895639A (en) * 2010-08-12 2010-11-24 宇龙计算机通信科技(深圳)有限公司 Method for displaying data and mobile terminal
CN103067522A (en) * 2013-01-10 2013-04-24 东莞宇龙通信科技有限公司 Data management method and system and mobile terminal based on cloud end server
CN103377078A (en) * 2012-04-11 2013-10-30 广州市地下铁道总公司 Real-time task scheduling method and system for vehicular ATP
CN105577410A (en) * 2014-10-16 2016-05-11 中兴通讯股份有限公司 Real-time terminal state refreshing method and apparatus thereof
CN108920267A (en) * 2012-05-24 2018-11-30 瑞萨电子株式会社 Task Processing Unit
CN109150596A (en) * 2018-08-08 2019-01-04 新智能源系统控制有限责任公司 A kind of SCADA system real time data dump method and device
CN110275767A (en) * 2019-06-26 2019-09-24 深圳前海微众银行股份有限公司 A kind of batch data processing method and processing device
CN111090699A (en) * 2019-12-13 2020-05-01 北京奇艺世纪科技有限公司 Service data synchronization method and device, storage medium and electronic device
CN112541041A (en) * 2020-12-23 2021-03-23 平安普惠企业管理有限公司 Data processing method, device, server and storage medium

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1241882A (en) * 1998-07-02 2000-01-19 日本电气株式会社 Method of subscriber data control in mobile communication network
CN101796487A (en) * 2007-08-10 2010-08-04 内特可力亚斯系统股份有限公司 Virtual queue processing circuit and task processor
CN101895639A (en) * 2010-08-12 2010-11-24 宇龙计算机通信科技(深圳)有限公司 Method for displaying data and mobile terminal
CN103377078A (en) * 2012-04-11 2013-10-30 广州市地下铁道总公司 Real-time task scheduling method and system for vehicular ATP
CN108920267A (en) * 2012-05-24 2018-11-30 瑞萨电子株式会社 Task Processing Unit
CN103067522A (en) * 2013-01-10 2013-04-24 东莞宇龙通信科技有限公司 Data management method and system and mobile terminal based on cloud end server
CN105577410A (en) * 2014-10-16 2016-05-11 中兴通讯股份有限公司 Real-time terminal state refreshing method and apparatus thereof
CN109150596A (en) * 2018-08-08 2019-01-04 新智能源系统控制有限责任公司 A kind of SCADA system real time data dump method and device
CN110275767A (en) * 2019-06-26 2019-09-24 深圳前海微众银行股份有限公司 A kind of batch data processing method and processing device
CN111090699A (en) * 2019-12-13 2020-05-01 北京奇艺世纪科技有限公司 Service data synchronization method and device, storage medium and electronic device
CN112541041A (en) * 2020-12-23 2021-03-23 平安普惠企业管理有限公司 Data processing method, device, server and storage medium

Similar Documents

Publication Publication Date Title
US20210133233A1 (en) Managing storage operations through an administrative assistant that processes natural language
CN111448550B (en) Network-accessible machine learning model training and hosting system
CN109558215B (en) Backup method, recovery method and device of virtual machine and backup server cluster
KR102025934B1 (en) Partitioning and moving ranges in a distributed system
CN109359091B (en) File management method, device, terminal and computer readable storage medium
KR20170133866A (en) Apparatus and method for data migration
US9378039B2 (en) Virtual machine storage replication schemes
US20240031120A1 (en) System and method for automatically synchronizing responses to conditions on devices
CN114661248B (en) Data processing method and device
CN106339221A (en) Clipboard control method and system based on mobile terminal
JP6193406B2 (en) Serialization for differential encoding
WO2024041433A1 (en) Data processing method and apparatus
CN113467719A (en) Data writing method and device
CN113515516A (en) Data processing method and device, computer equipment and computer storage medium
US8726147B1 (en) Systems and methods for restoring web parts in content management systems
JP5729390B2 (en) Application configuration system, method, and program
US20230281187A1 (en) Method for keeping data consistent across different storage systems, computing device, and storage medium
WO2021077862A1 (en) File synchronization method and device
CN114443216A (en) Cloud platform mirror image management method and management system
CN109325057B (en) Middleware management method, device, computer equipment and storage medium
JP5659248B2 (en) Communication terminal and synchronization method
CN115604667B (en) Message sending method, device, computer equipment and storage medium
JP7106870B2 (en) Program update system and program update method
CN113311995A (en) Cloud hard disk preparation method and device, computer equipment and storage medium
CN113760349A (en) Mirror image upgrading method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information
CB02 Change of applicant information

Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing.

Applicant after: Douyin Vision Co.,Ltd.

Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing.

Applicant before: Tiktok vision (Beijing) Co.,Ltd.

Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing.

Applicant after: Tiktok vision (Beijing) Co.,Ltd.

Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing.

Applicant before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd.