WO2022242372A1 - 对象处理方法、装置、计算机设备和存储介质 - Google Patents

对象处理方法、装置、计算机设备和存储介质 Download PDF

Info

Publication number
WO2022242372A1
WO2022242372A1 PCT/CN2022/086091 CN2022086091W WO2022242372A1 WO 2022242372 A1 WO2022242372 A1 WO 2022242372A1 CN 2022086091 W CN2022086091 W CN 2022086091W WO 2022242372 A1 WO2022242372 A1 WO 2022242372A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
current
conflict
stage
interaction request
Prior art date
Application number
PCT/CN2022/086091
Other languages
English (en)
French (fr)
Inventor
潘安群
熊亮春
李纲
叶盛
Original Assignee
腾讯科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技(深圳)有限公司 filed Critical 腾讯科技(深圳)有限公司
Priority to EP22803697.6A priority Critical patent/EP4239492A4/en
Priority to JP2023523281A priority patent/JP7566146B2/ja
Publication of WO2022242372A1 publication Critical patent/WO2022242372A1/zh
Priority to US18/062,308 priority patent/US20230098963A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2365Ensuring data consistency and integrity
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2308Concurrency control
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2358Change logging, detection, and notification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/52Program synchronisation; Mutual exclusion, e.g. by means of semaphores
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/52Program synchronisation; Mutual exclusion, e.g. by means of semaphores
    • G06F9/524Deadlock detection or avoidance

Definitions

  • the present application relates to the field of network technology, in particular to an object processing method, device, computer equipment and storage medium.
  • the interaction with the object is realized through strong synchronization, that is, specific computing nodes are allowed to execute interactive statements in a centralized manner. This will cause other computing nodes to wait for the execution result of the centrally controlled computing node to perform subsequent processing. Lead to inefficiency in object interaction processing.
  • An object processing method executed by a computer device, the method comprising: receiving an object interaction request for a target object; in response to the object interaction request, acquiring the current change stage of the current change operation for the target object;
  • the current change operation includes multiple operation steps, and the current change operation is divided into multiple change stages according to the conflict operation type corresponding to each of the operation steps, and the conflict operation type is the conflict operation corresponding to the operation step type; according to the target operation type corresponding to the object interaction request and the current related operation type corresponding to the current change stage, determine the conflict check result corresponding to the object interaction request;
  • the target operation type is the object the type of the target interaction operation requested by the interaction request; and when the result of the conflict check is that there is no conflict, executing the target interaction operation on the target object according to the object interaction request.
  • An object processing device executed by computer equipment, comprising: an interaction request receiving module, configured to receive an object interaction request for a target object; a change stage acquisition module, configured to respond to the object interaction request, and acquire the The current change stage where the current change operation of the target object is located; the current change operation includes multiple operation steps, and the current change operation is divided into multiple change stages according to the conflict operation type corresponding to each operation step.
  • the conflict operation type is the type of conflict operation corresponding to the operation step; the conflict checking module is configured to determine the corresponding operation type according to the target operation type corresponding to the object interaction request and the current related operation type corresponding to the current change stage.
  • the conflict verification result corresponding to the object interaction request; the target operation type is the type of the target interaction operation requested by the object interaction request; and the operation execution module is configured to be used when the conflict verification result indicates that there is no conflict , execute the target interaction operation on the target object according to the object interaction request.
  • a computer device comprising a memory and one or more processors, the memory having stored therein computer readable instructions that, when executed by the one or more processors, cause the one or more A processor executes the steps in the methods described in the various embodiments of the present application.
  • One or more computer-readable storage media having stored thereon computer-readable instructions that, when executed by one or more processors, cause the one or more processors to Execute the steps in the methods described in the various embodiments of the present application.
  • a computer program product comprising computer readable instructions stored in a computer readable storage medium; read by one or more processors of a computer device from the computer readable storage medium
  • the computer readable instructions are executed by one or more processors, so that the computer device executes the steps in the methods described in the various embodiments of the present application.
  • Fig. 1 is an application environment diagram of an object processing method in an embodiment
  • Fig. 2 is an application environment diagram of an object processing method in another embodiment
  • Fig. 3 is a schematic flow chart of an object processing method in an embodiment
  • Fig. 4 is a schematic diagram of a state transition matrix in an embodiment
  • Fig. 5 is a schematic flow chart of the change operation in an embodiment
  • Fig. 6 is a schematic flow chart of an object processing method in another embodiment
  • Fig. 7 is a schematic flow chart of preparation operation in an embodiment
  • Fig. 8 is a schematic flow chart of the finishing operation in one embodiment
  • FIG. 9 is a schematic flow diagram of executing a data definition language statement in an embodiment
  • Fig. 10 is a structural block diagram of an object processing device in an embodiment
  • Figure 11 is a diagram of the internal structure of a computer device in one embodiment.
  • the object processing method provided in this application can be applied to the application environment shown in FIG. 1 .
  • the application environment includes a server 102 , a first terminal 104 , a second terminal 106 and a database 108 .
  • the first terminal 104 and the second terminal 106 respectively communicate with the server 102 through the network
  • the database 108 is a database for storing data of each terminal, and can communicate with the server and each terminal through the network.
  • the first terminal 104 performs the current change operation on the target object in the database 108.
  • the current change operation corresponds to multiple change stages, and the first terminal 104 sends the execution status of each change stage when executing each change stage to the server.
  • the server 102 When receiving the object interaction request for the target object sent by the second terminal 106, the server 102 obtains the current change stage corresponding to the current change operation of the first terminal 104 through the execution status of each change stage sent by the first terminal 104, according to The target operation type corresponding to the object interaction request and the current related operation type corresponding to the current change stage, determine the conflict verification result corresponding to the object interaction request, and stop executing the target interaction operation according to the object interaction request when the conflict verification result is a conflict , when the result of the conflict verification is that there is no conflict, perform the target interaction operation on the target object according to the object interaction request, and send the conflict verification result to the second terminal 106 .
  • the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication Cloud servers for basic cloud computing services such as middleware services, domain name services, security services, CDN (Content Delivery Network, content distribution network), and big data and artificial intelligence platforms.
  • Terminals including the first terminal and the second terminal
  • the terminal and the server may be connected directly or indirectly through wired or wireless communication, which is not limited in this application.
  • the first terminal 104 and the second terminal 106 may also be implemented by one terminal device, and in addition, the terminal may also be replaced by a server.
  • the terminal that interacts with the target object can be implemented by a computing engine node (SQL Engine, also called a computing node or node), and mainly completes the SQL (Structured Query Language, Structured Query Language) layer computing function . It should be noted that in the embodiment of the present application, there may be at least two.
  • the object processing method provided in this application can also be applied to the application environment shown in FIG. 2 .
  • the application environment includes a terminal 202 , a computing node cluster 204 , and a database 206 .
  • the computing node cluster 204 may include at least one computer device, which may be called a computing node, and these computing nodes may be implemented by servers or interrupts.
  • each computing node in the computing node cluster 204 may have the interaction authority on the same object.
  • Two computing nodes, the first computing node 2042 and the second computing node 2044 are shown in FIG. 2 . In an actual application scenario, fewer or more computing nodes may be included.
  • the first computing node 2042 and the second computing node 2044 respectively communicate with the database 206 through the network, and the database 206 is a database for storing data of each computing node.
  • the first computing node 2042 executes the current change operation on the target object in the database 206.
  • the current change operation corresponds to multiple change stages.
  • the state is saved to database 206 .
  • the second computing node 2044 When the second computing node 2044 receives the object interaction request for the target object sent by the terminal 202, it obtains the execution status of each change stage under the current change operation of the first computing node 2042 from the database, and determines to obtain the first Calculate the current change stage corresponding to the current change operation of the node 2042, and determine the conflict check result corresponding to the object interaction request according to the target operation type corresponding to the object interaction request and the current related operation type corresponding to the current change stage, wherein the target operation Type, which is the type of the target interaction operation requested by the object interaction request; when the conflict verification result shows that there is a conflict, stop executing the target interaction operation according to the object interaction request; Executes the target interaction operation on the target object.
  • Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to realize data calculation, storage, processing, and sharing.
  • Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on cloud computing business model applications. It can form a resource pool and be used on demand, which is flexible and convenient. Cloud computing technology will become an important support.
  • the background services of technical network systems require a lot of computing and storage resources, such as video websites, picture websites and more portal websites.
  • each item may have its own identification mark in the future, which needs to be transmitted to the background system for logical processing.
  • Data of different levels will be processed separately, and all kinds of industry data need to be powerful.
  • the system backing support can only be realized through cloud computing.
  • the storage resources required by the background service can be realized through the database.
  • the database in short, can be regarded as an electronic filing cabinet—a place where electronic files are stored, and users can perform operations such as adding, querying, updating, and deleting data in files.
  • the so-called “database” is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application program.
  • a database management system (English: Database Management System, referred to as DBMS) is a computer software system designed for managing databases, and generally has basic functions such as storage, interception, security, and backup.
  • Database management systems can be classified according to the database model it supports, such as relational, XML (Extensible Markup Language, Extensible Markup Language), or according to the type of computer supported, such as server clusters, mobile phones; Or classify according to the query language used, such as: SQL, XQuery; or classify according to the focus of performance impulse, such as the largest scale, the highest operating speed; or other classification methods. Regardless of the taxonomy used, some DBMSs are capable of cross-categorying, for example, supporting multiple query languages at the same time.
  • the data generated during the operation of each embodiment can be stored on the block chain.
  • the current change stage of the current change operation of the target object is recorded in the blockchain; when receiving an object interaction request, the terminal or server obtains the current change stage from the blockchain to determine the current relevant operation corresponding to the current change stage Type, determine the conflict verification result corresponding to the object interaction request according to the target operation type and the current related operation type; when it is determined that there is a conflict, stop executing the target interaction operation according to the object interaction request; when it is determined that there is no conflict, continue to execute according to the object interaction request Target interactive operation, and record the status information of each stage in the interactive operation process to the blockchain.
  • the conflict verification results can also be recorded in the blockchain.
  • the terminal or server can obtain the status information of each stage in the interactive operation process from the blockchain.
  • the target operation type, current related operation type, and conflict verification results generated during the operation can all be recorded in the blockchain.
  • an object processing method is provided, which is applied to a computer device.
  • the computer device may be a server or a terminal.
  • This embodiment is described by taking the method applied to a server as an example. It can be understood that the method can also be applied to a system including a terminal and a server, and is implemented through interaction between the terminal and the server.
  • the method includes the following steps:
  • the object is the subject targeted by the network interaction, which may be at least one of network equipment, data in the network equipment, data in the database, and the like.
  • the target object may be at least one item of data or metadata stored in a database.
  • the target object may be a Schema (an abstract collection of metadata) in the database, for example: at least one of the following items in the database: tables, columns, indexes, constraints, stored procedures or functions, and so on.
  • the number of target objects may be at least one. When there are at least two target objects, conflict checks can be performed for each target object, and then targeted to stop or continue to perform target interaction operations on each target object according to the object interaction request.
  • the interaction may be at least one of operations such as changing or querying the target object.
  • the change may be at least one of: new creation, modification, deletion, insertion, or update. Specifically, inserting one column into a table can be considered as one change operation, and inserting two columns can be considered as two change operations.
  • an object interaction request is a request for a specific interaction with a target object.
  • each object interaction request may correspond to at least one object interaction operation, and after the object interaction request is determined, the corresponding object interaction operation may be determined to obtain a target interaction operation.
  • An object interaction request can correspond to at least one target interaction operation. For example, for the following object interaction request: alter table t add a int, add b int, the corresponding target interaction operation is to add two columns a and b to table t, Each operation that adds a column can be thought of as a target interaction operation.
  • the server may determine a target change operation corresponding to the object interaction request.
  • the object interaction operation can be DML (Data Manipulation Language, Data Manipulation Language), DDL (Data Definition Language, Data Definition Language) or DCL (Data Control Language, Data Control Language), etc. at least one of the .
  • DML can be commands such as select, update, insert, and delete. These 4 commands are used to operate the data in the database;
  • DDL DDL has more command types than DML.
  • DDL is mainly used to define or change the table (table) structure, data types, links and constraints between tables and other initialization work, mostly used when creating tables;
  • DCL is the database control function, is Statements used to set or change database user or role permissions, including grant, deny, or revoke statements.
  • languages such as DML, DDL, or DCL are operations for changing objects in the database, so they can be called object change operations, and corresponding requests can be called object change requests.
  • object interaction operations can also be subdivided, for example: alter can also be subdivided into: copy (copy), inplace (replace) or online (online) and other operations, these operations can also be divided into multiple change stages based on operation steps .
  • the object interaction request may be a command statement sent by the terminal received by the server, for example, the server receives a DDL statement sent by the terminal, and the DDL statement is an object interaction request.
  • the server may receive the object interaction request for the target object sent by the terminal.
  • the current change stage of the current change operation for the target object includes multiple operation steps, and the current change operation is divided into multiple change stages according to the conflict operation type corresponding to each operation step , the conflict operation type is the type of the conflict operation corresponding to the operation step.
  • the change operation is an operation of changing the target object, which may be an operation of changing the content and structure of the target object, and specifically may be at least one of creating, modifying, deleting, inserting, or updating.
  • the current change operation can be the change operation that a specific computing node is performing on the target object when the object interaction request is received, or it can be the change operation that the specific computing node performs on the target object before receiving the object interaction request or after the set time period.
  • a change operation consists of several change phases. Wherein, the change stage is an execution stage subdivided by executing a certain change operation.
  • Each change stage can contain at least one operation step, and each operation step in the same change stage corresponds to the same type of conflicting operation, that is, if computing node C sends an object interaction request during the execution of the same change stage, the response from the server only one. For example, computing node C is triggered to perform rollback or continue execution.
  • the execution priority of the current change operation may be higher than or equal to the target operation corresponding to the object interaction request. Further, the execution priority of each change stage of the current change operation may also be higher than or equal to the target operation corresponding to the object interaction request. stages of interaction.
  • the current change stage can be the change stage that the specific computing node is executing on the target object when the object interaction request is received, or it can be the specific computing node before or after the set time period is received for the object interaction request The phase of the change performed on the target object.
  • one operation may include multiple operation steps, that is, one operation is divided into multiple operation steps for execution, and the function to be achieved by the operation can be completed by executing the multiple operation steps corresponding to the operation.
  • the operation step is the operation unit corresponding to the operation.
  • an operation may include establishing a handshake connection, accessing an interface, obtaining meta information, verifying meta information, updating status flags, reading a certain data record, comparing data records, or modifying data records, etc. at least one of the steps. These operation steps are executed in a certain sequence to complete the corresponding change operation.
  • the conflicting operation corresponding to the operation step is an interactive operation that conflicts with the operation step, that is, if a certain operation step runs in parallel with the conflicting operation, it may cause information confusion in the information corresponding to the target object. Therefore, the server should prevent the operation Steps run in parallel with the corresponding conflicting actions.
  • the conflict operation type is the type of conflict operation corresponding to the operation step. Wherein, the conflict operation may include multiple types, and may be at least one of new, modify, delete, insert, update, or query.
  • the current change operation on the target object may be a change operation performed on the target object by the target computing node.
  • the target computing node may be other computer equipment different from the aforementioned server.
  • the process of dividing the current change operation into change stages may be: 1.
  • the server automatically divides the change operation according to a certain method, that is, the server automatically divides the current change operation into multiple change stages according to the conflicting operation type; 2. , It can also be manually divided in advance.
  • the server can determine whether each operation step can run in parallel with other computing nodes in the computing node cluster.
  • the operation type corresponding to the node is used as the conflicting operation type of the corresponding operation step.
  • continuous operation steps under the same conflicting operation type can be divided into the same change phase, and operation steps under different conflicting operation types can also be divided into different change phases.
  • the computing node cluster is a cluster composed of computing nodes, and these computing nodes can interact with the target object.
  • the target object is an object in the target database
  • the target database is a database corresponding to the server, and terminals and servers capable of accessing the database are determined as computing nodes, and these computing nodes form a computing node cluster.
  • the server can set a status record table, and the status record table can be expressed as ddl_job or ddl_jobs.
  • the state record table records the execution status of each change stage in each change operation for the target object.
  • the initial state corresponding to each change stage can be unexecuted, and the computing node can change the state from unexecuted to completed after each change stage is executed. implement.
  • the server can access the state record table to locate the target state record corresponding to the current change operation, and determine the first unexecuted change stage in the target state record as the current change according to the sequence of change stages stage.
  • the state record of the latest completed change stage can be obtained from the state record table, and then based on the state record, the change stage before the execution failure can be restored. Re-execute the change phase that failed to execute.
  • the state recording table may be stored in the server. It can also be stored in the computing node that performs the current change operation, and can also be stored in a server and a database that can be accessed by each computing node.
  • the current related operation type includes a current conflicting operation type or a current parallel operation type.
  • the target operation type is the type of the target interaction operation, which may be at least one of create, modify, delete, insert, update, or query.
  • the target operation type corresponding to the object interaction request can be at least one type.
  • each target operation type can be compared with the current related operation type corresponding to the current change stage, and then the conflict verification result can be obtained. .
  • the current conflicting operation type is an operation type that is not allowed to be executed synchronously with the current change stage
  • the current parallel operation type is an operation type that is allowed to be executed synchronously with the current change stage
  • the current conflicting operation type may include at least one conflicting operation type
  • the current parallel operation type can also contain at least one parallel operation type.
  • the current conflicting operation type and the current parallel operation type can also be determined as the current related operation type.
  • the current conflicting operation type and the current parallel operation type may be recorded in a corresponding operation type record table according to predetermined operation types.
  • the operation type record table may include a conflict operation type record table and a parallel operation type record table.
  • the server determines the conflict operation type record table corresponding to the current change operation, and determines the conflict operation type recorded in the conflict operation type record table as the current conflict operation type.
  • the current parallel operation type can be determined according to the predetermined parallel operation type record table of each change operation.
  • the server determines the parallel operation type record table corresponding to the current change operation, and determines the parallel operation type recorded in the parallel operation type record table as the current parallel operation type.
  • the object interaction request is an object change request
  • a corresponding target change operation is performed on the target object. For example: when a DDL statement is received, the command corresponding to the DDL statement is executed on the target object.
  • the target operation type corresponding to the object interaction request and the current related operation type corresponding to the current change stage are allowed to run in parallel, and the target interaction operation can be executed according to the object interaction request.
  • the execution of the target interaction operation according to the object interaction request is stopped.
  • the target operation type corresponding to the object interaction request and the current related operation type corresponding to the current change stage are not allowed to run in parallel, and the execution of the target interaction operation according to the object interaction request can be stopped.
  • the change operation by subdividing the change operation into change stages, conflict checks can be performed in each change stage, operation stages without conflicts can be run in parallel, and operation stages with conflicts can be stopped at the same time, which can ensure operation
  • the atomicity can greatly reduce the impact of change operations on other interactive operations, and effectively improve the efficiency of object interaction processing.
  • the current change operation is subdivided into change stages, which refines the comparison between the target interaction operation and the current change operation.
  • the target interaction operation does not need to wait until all stages of the change operation are fully executed, and this method will not affect the production environment. , which can minimize the impact of the change operation on other interactive operations, and is imperceptible to the user who triggers the object interaction request.
  • obtaining the current change stage of the current change operation on the target object includes: obtaining the current operation step corresponding to the current change operation on the target object; determining the change stage corresponding to the current operation step as the target object The current change stage where the current change operation is located, wherein the continuous operation steps of different conflicting operation types are divided into different change stages.
  • the current operation step can be the operation step that the specific computing node is performing on the target object when receiving the object interaction request, or it can be the operation step that the specific computing node performs on the target object before receiving the object interaction request or after the set time period. Action steps performed by the target object.
  • the corresponding relationship between the operation steps and the change stages may be preset, or the change stages corresponding to the operation steps may be determined immediately.
  • the server determines the current change stage corresponding to the current operation step according to the correspondence between the operation step and the change stage.
  • the current change stage is determined based on the current operation step corresponding to the current change operation, and then the conflict check is performed on the object interaction request based on the current change stage.
  • the accurate current change stage can be determined through detailed operation steps, and then reliable conflict verification results can be obtained.
  • the step of dividing the current change operation into stages includes: determining the current operation step sequence corresponding to the current change operation, the current operation step sequence includes a plurality of operation steps arranged according to the operation order; determining the current operation step sequence The conflict operation type corresponding to each operation step in the operation step; the current operation step sequence is divided according to the conflict operation type corresponding to the operation step, and multiple change stages corresponding to the current change operation are obtained. Among them, the continuous operation steps of the same conflict operation type Partitioned in the same change phase.
  • the operation step sequence is a sequence obtained by sorting a plurality of operation steps according to the order of execution. Performing at least two sequentially adjacent operation steps can be regarded as consecutive operation steps. That is, for adjacent operation steps, if the corresponding conflicting operation types are different, they are divided into different change stages.
  • the change operation includes multiple operation steps.
  • the new operation can include the following consecutive steps: 1. Obtain meta information, 2. Verify meta information, 3. Add new table data to the data dictionary, where the data dictionary can be stored in the database 4. Status identification update, etc. These operating steps constitute the current operating step sequence. Each operation step may have different conflict states in the interactive operation with other computing nodes during execution.
  • each operation step of the insert operation can be run in parallel with the operation steps of obtaining meta-information, verifying meta-information, and status identification update in the new operation, that is, there is no conflict, and each operation step of the insert operation
  • the operation steps and the operation steps of adding new table data to the data dictionary in the new operation cannot be run in parallel, and the insert operation needs to be rolled back, that is, there is a conflict.
  • the conflicting operation types corresponding to the consecutive operation steps 1 and 2 are the same, therefore, these two operation steps can be divided into the same change stage as the meta information verification stage, and the conflicting operations corresponding to operation steps 2 and 3 The operation types are different. Therefore, these two operation steps can be divided into different change stages.
  • operation steps 3 and 4 are divided into data adding phase and identification updating phase respectively. Accordingly, the operation step sequence corresponding to the new operation is divided into the meta information verification phase, the data addition phase and the identification update phase.
  • the number of operations to be compared can be more, and can also be all interactive operations on the same object in the computing node cluster.
  • the change operation is divided into change stages according to the conflict operation type corresponding to each operation step in the sequence of operation steps, so that continuous operation steps of the same conflict operation type are divided into the same change stage, and different conflict operations are divided into different stages.
  • Types of continuous operation steps are divided into different change stages, and the division of change stages can be accurately realized based on the conflicting operation types corresponding to the operation steps.
  • determining the conflict verification result corresponding to the object interaction request includes: extracting the operation type identifier from the object interaction request, according to The extracted operation type identifies the target operation type corresponding to the object interaction request; compares the target operation type with the current conflict operation type, and determines that the conflict check result exists when the current conflict operation type includes the target operation type according to the comparison result conflict.
  • the computing node when it sends the object interaction request, it may carry the type identifier of the target operation type.
  • the server can obtain the target operation type by extracting the type identifier from the information carried in the object interaction request.
  • the target operation type may be compared with the current conflicting operation type one by one. When there is a conflict operation type consistent with the target operation type, it is determined that the current conflict operation type includes the target operation type, and at this time, the conflict check result will be determined as a conflict; when there is no conflict operation type consistent with the target operation type, It is determined that the current conflicting operation type does not include the target operation type, and at this time, it will be determined that there is no conflict as a result of the conflict check.
  • the conflict check is performed based on the comparison result between the target operation type and the current conflict operation type, and then the conflict check result can be quickly obtained, effectively improving the efficiency of object processing.
  • determining the conflict verification result corresponding to the object interaction request includes: extracting the operation type identifier from the object interaction request, according to The extracted operation type identifies the target operation type corresponding to the object interaction request; compares the target operation type with the current parallel operation type, and determines that the conflict check result is There is a conflict.
  • the target operation type may be compared with the current parallel operation type one by one. When there is no conflict operation type consistent with the target operation type, it is determined that the current parallel operation type does not include the target operation type. At this time, the conflict check result will be determined as a conflict; similarly, when there is a conflict consistent with the target operation type When the operation type is selected, it is determined that the current parallel operation type includes the target operation type, and at this time, it is determined that there is no conflict as a result of the conflict check.
  • the conflict check is performed based on the comparison result between the target operation type and the current parallel operation type, so that the conflict check result can be quickly obtained, and the efficiency of object processing can be effectively improved.
  • the operation type recording table can be implemented by a state transition matrix, which can record the transition possibilities between the various states of the target object. If the transition is possible, it is recorded as allowing the transition, and if it is not possible to transition , the conversion is logged as disallowed. Further, the conflict check result corresponding to the object interaction request may be determined according to the state transition matrix.
  • the state of the target object can be the data reading and writing state of the target object, which can be public (public), readable (read), writable (write), unreadable (no write), unwritable (no read), waiting (waiting) or delete only (delete only), etc. at least one state. Among them, open refers to the state where read, write and delete operations can be performed.
  • each operation type may correspond to a different state of the target object.
  • the target state of the target object may be determined according to the target operation type, and the current state of the target object may be determined according to the operation type of the current change operation. Specifically, if the current change operation is not completed, the current state may be the state before the current change operation is performed, and if the current change operation is completed, the current state may be the state after the current change operation is performed.
  • the expected state of the target object by the target operation type may be determined as the target state of the target object.
  • each change stage can also be stateful, and the current state of the target object can also be determined according to the execution state of the current change stage.
  • the specific process please refer to the above-mentioned process of determining the current state according to the current change operation, which will not be repeated here. repeat.
  • the target state corresponding to the target object after the target object is interacted with according to the target operation type is determined, and when it is determined based on the state transition matrix that the target object is allowed to transition from the current state to the target state, the conflict check result is determined as There is no conflict; otherwise, the conflict verification result corresponding to the object change request is determined as conflict exists.
  • the state transition matrix may be as shown in FIG. 4 .
  • Figure 4 records whether it is allowed to convert the state of the column item to the state of the row item, wherein, the arrow in the table indicates the conversion direction, Y indicates that it is allowed, and N indicates that it is not allowed.
  • the current state of the target object is new-public, that is, the current change operation is a new operation, and the corresponding The state is public; the target object of the target object is modify-public, that is, the target interaction operation is a modification operation, and the corresponding state is public when the modification operation is executed.
  • This transition is allowed by querying the information in the state transition matrix in FIG. 4 . Therefore, the result of the conflict check is determined to be that there is no conflict.
  • the method further includes: when the result of the conflict check is that there is a conflict, stopping performing the target interaction operation on the target object according to the object interaction request.
  • a rollback operation may also be performed to restore the state before receiving the object interaction request. Afterwards, the execution status of the current change operation can be monitored in real time. If the current change operation is completed, the latest state of the target object after the current change operation has been changed can be obtained, and the object interaction request can be responded to again, and then the target interaction operation can be executed based on the latest state of the target object. .
  • the interaction process can be performed based on the latest state of the target object, preventing differences in the state of the target object acquired by different computing nodes.
  • performing the target interaction operation on the target object according to the object interaction request includes: when the result of the conflict verification is that there is no conflict, determining the target object in response to the object interaction request The target operation step sequence corresponding to the operation type; perform the target interaction operation on the target object according to the target operation step sequence.
  • the operation step sequences corresponding to various operation types may be recorded through a step sequence recording table.
  • the operation step sequence corresponding to the operation type identifier is determined from the step sequence recording table based on the operation type identifier of the target operation type to obtain the target operation step sequence.
  • the target operation step sequence corresponding to the target operation type is determined, and then the target interaction operation is performed on the target object according to the target operation step sequence, and the target interaction operation with the object interaction request can be performed on the target object, ensuring the reliability of the object interaction request response.
  • performing the target interaction operation on the target object according to the sequence of target operation steps includes: determining the first operation stage corresponding to the target object, where the first operation stage is the next operation stage corresponding to the object interaction request; when the first operation stage When the corresponding stage priority satisfies the priority execution condition, an operation lock request for the target object is sent; when it is determined that the lock response information for the operation lock request is received, the operation steps corresponding to the first operation stage are executed.
  • the operation locking request is a request for triggering operation locking.
  • an operation lock request is sent to computing nodes that are interacting with the target object, triggering these computing nodes to add corresponding object locks to lock in the current operation on the target object until unlocked.
  • the lock response information may be the response information returned by the computing node to the server after adding the object lock, that is, the feedback that the lock application is successful.
  • the server receives the lock response information, it may determine that the corresponding computing node has added the corresponding object lock.
  • the process of determining whether the phase priority corresponding to the first operation phase meets the priority execution condition may be: obtaining the priority of each phase in each interaction operation that interacts with the target object, The corresponding stage priority is compared with the previous stage priorities, and when the stage priority corresponding to the first operation stage is the highest, it is determined that the stage priority corresponding to the first operation stage satisfies the priority execution condition.
  • the implementation process of sending the operation lock request for the target object may be: sending the operation lock request for the target object to other computing nodes in the computing node cluster, so that these computing nodes return locking response information to the server.
  • an operation lock request may be sent to all computing nodes in the computing node cluster, which may be regarded as applying for a global lock at this time.
  • an operation lock request can also be sent to some computing nodes in the computing node cluster, which can be regarded as applying for a local lock.
  • the operation steps corresponding to the first operation stage may be executed when receiving the locking response information of some computing nodes; or When the locking response information of all computing nodes is received, the operation steps corresponding to the first operation stage are executed.
  • executing the target interaction operation on the target object according to the sequence of target operation steps includes: determining the next operation stage corresponding to the target object to obtain the first operation stage; sending an operation lock request to the computing node cluster, so that each computing node cluster When the computing node is compatible with the operation lock request, add the object lock and return the lock response information; when receiving the lock response information returned by each computing node in the computing node cluster, execute the first operation stage; when the first operation stage is completed Send an operation unlock request to the computing node cluster at any time, so that each computing node in the computing node cluster unlocks the object lock and continues to run; return to the next step of changing the target object according to the object change request, until the object change request The execution of each corresponding operation change stage is completed.
  • the target interactive operation is divided into multiple small operation phases for execution. It is not necessary to apply for locks for all operation phases, but to apply for locks at the beginning of the operation phase that needs to be controlled. After each operation phase is executed Just release the lock.
  • the data in table A is stored in the data table, and table A is being deleted based on an object change operation. Table A has been deleted and is currently in the process of deleting the data in table A in the data table. The change operation of creating a new table with the same name, since table A has been deleted, there will be no conflict between the newly created table and the deleted table, and there will be no conflict with the operation of deleting the data table, so the process of creating a new table does not need to apply for a lock .
  • the entire computing node cluster can appear as a whole to the outside world, and the server needs to apply for a global lock before the next stage of DDL may cause rollback of other DML statements. Strong synchronization is possible at this point.
  • the specific process of applying for a global lock can be as follows:
  • the server first sends a lock request to all nodes in the computing node cluster and waits for a period of time;
  • the server checks whether other computing nodes have feedback that they are already holding incompatible locks. If so, wait and go to 3. If there is no feedback of holding a lock or a lock response message is received, it means that it can advance to the next stage , enter 4;
  • the server waits for the lock response information of each computing node
  • the server advances the state of the target object. Due to the advancement of the state, other computing nodes may roll back accordingly if they did not send feedback in the previous stage.
  • the server sends a lock release message to the feedback computing node, and continues to perform the above steps in a loop until the DDL statement is completed.
  • the server requests the operation lock at each operation stage, so that each operation stage can be used as an independent operation unit, and then the conflict check of the smallest unit can be realized, and the waiting process of the computing nodes can be reduced as much as possible.
  • the object processing method further includes: determining a second operation stage corresponding to the target object, where the second operation stage is the next operation stage corresponding to the first operation stage; when the stage priority corresponding to the second operation stage satisfies When the condition is prioritized, the object lock requested by the operation lock request is maintained, and the operation steps corresponding to the second operation stage are executed.
  • sending the operation lock request for the target object includes: determining the operation content level of the operation content corresponding to the first operation stage; When the content level is the object level, it is determined that the phase priority corresponding to the first operation phase satisfies the priority execution condition, and an operation lock request for the target object is sent.
  • the operation content is the content operated on the target object by the interactive operation
  • the operation content may be the content of the object level or the content of the data level.
  • the operation content can be at the object level, for example: operations on the table structure, data type, links and constraints between tables, etc.
  • the operation content can be at the data level, for example: Insert a record into the table, modify the data of a record in the table, etc.
  • the content-level interactive operation affects the target object more than the data-level interactive operation affects the target object.
  • the stage priority corresponding to the operation stage is determined based on the operation content level, and then whether to send the operation lock request is determined.
  • Content-level interactive operations can be executed first, while data-level interactive operations are executed later, which can effectively prevent the influence of content-level interactive operations overriding data-level interactive operations after data-level interactive operations are executed, resulting in invalid operations. In this way, the efficiency of object processing can be effectively improved.
  • the stage priority corresponding to the first operation stage can also be determined according to the target operation type.
  • the stage priority corresponding to each operation stage of the target interactive operation is determined to be Priority conditions are met.
  • the operation content level of the target interactive operation is the object level, it is determined that the type of the target operation meets the preset type condition, and it is determined that the stage priorities corresponding to each operation stage of the target interactive operation meet the priority execution condition.
  • the target operation type is the operation type corresponding to the DDL statement
  • the stage priorities corresponding to each operation stage of the target interactive operation meet the priority execution condition, that is, the execution of the DDL statement is prioritized over statements such as DML.
  • the object processing method further includes: determining the target object version identifier corresponding to the object interaction request; acquiring the current object version identifier corresponding to the target object after the target object is changed according to the current change operation; The identifier is compared with the current object version identifier, and when the comparison is inconsistent, stop responding to the object interaction request.
  • the object interaction request carries the version identifier of the target object
  • the server determines the version identifier of the target object carried in the request to obtain the target object version identifier.
  • the target object corresponds to a version identifier.
  • the version identifier of the target object will change. For example, each time a change is made to the target object, the version identifier of the target object will be +1, and the target object can be obtained accordingly. The corresponding current object version identifier.
  • the object interaction request it is determined whether to respond to the object interaction request by comparing the version identifier of the target object, and only when the target object version identifier is consistent with the current object version identifier, the object interaction request is responded. It can make the interactive operation on the target object be carried out based on the latest state of the target object, prevent differences in the content of the target object obtained by different computing nodes, and ensure that all computing nodes in the computing node cluster have consistent data on the target object sex.
  • the object interaction request is an object change request
  • executing the target interaction operation according to the object interaction request includes: when the next object change stage is an information verification stage, obtaining basic interaction information corresponding to the object interaction request; The basic information is verified; when the interactive basic information is verified, the state of the target object is changed.
  • the object interaction request is an object change request
  • executing the target interaction operation according to the object interaction request includes: when the next object change stage is the version identification writing stage, perform a corresponding state change operation on the target object; obtain The target state version identifier corresponding to the target object after the state change operation; write the target state version identifier into the storage space, so that when the storage node in the storage space receives a new object interaction request for the target object, determine the new The state version identifier carried in the object interaction request, and when the state version identifier carried in the object interaction request does not match the target state version identifier, stop responding to the new object interaction request.
  • the storage node can also be called a storage engine node (TD Store), which can be a computer device that can run in the storage layer, and mainly completes transaction and storage functions.
  • TD Store storage engine node
  • the object interaction request is an object change request
  • executing the target interaction operation according to the object interaction request includes: when the next object change stage is the state broadcast stage, perform a corresponding state change operation on the target object;
  • the operation generates state change broadcast information;
  • the state change broadcast information is used to trigger each computing node in the computing node cluster to obtain the target object after the state changing operation.
  • the state change broadcast information is broadcast information generated when the state of the target object changes.
  • Each computing node can obtain the latest state and latest version information corresponding to the target object from the state record table after receiving the state change broadcast information.
  • state change broadcasting can be represented by global schema version (global object version identifier, GLSV) and so on.
  • the object change request is responded to through different change stages, and the execution sequence of these change stages can be adjusted according to the actual situation.
  • the number and order of change stages under different target operation types can be different. In this way, the response to the object change request can be realized through multiple independent change stages, ensuring the atomic control of the change operation.
  • determining the conflict verification result corresponding to the object interaction request includes: obtaining a status record table from the storage space, status The record table is used to record the state information corresponding to each object change operation; when there is no state information of the conflicting object change operation in the state record table, according to the target operation type corresponding to the object interaction request and the current relevant operation type corresponding to the current change stage , to determine the conflict verification result corresponding to the object interaction request; the conflicting object change operation is a change operation that conflicts with the object interaction request.
  • the conflict check performed in this embodiment can be considered as a conflict check between interactive operations, and can be considered as a first-level conflict check.
  • the object interaction request is an object change request
  • when there is no conflicting object change operation state information in the state record table, according to the target operation type corresponding to the object interaction request and the current relevant operation type corresponding to the current change stage , to determine the conflict verification result corresponding to the object interaction request including: obtaining the object ID of the target object; sending an object registration request to the cluster management node based on the object ID, so that the cluster management node responds to the object registration request and generates uniqueness based on the object ID Registration number; When the unique registration number sent by the cluster management node is obtained, the conflict verification result corresponding to the object interaction request is determined according to the target operation type corresponding to the object interaction request and the current related operation type corresponding to the current change stage.
  • the conflict check performed in this embodiment can be regarded as a conflict check in an interactive operation, and can be regarded as a second-level conflict check.
  • each change stage of the current change operation may correspond to an operation type, and these operation types may be determined based on the state of the pre-configured target object, for example: for the operation stage in the delete-only state, the operation type may be is the delete operation type; for the operation phase in the write state, the operation type can be a write operation type; for the operation phase in the read state, the operation type can be a read operation type; and so on.
  • determining the conflict check result corresponding to the object interaction request includes: when the current change stage is the blocking operation information writing stage , determine that the current parallel operation type corresponding to the current change stage is the delete operation type; the blocking operation information writing stage is used to write the current object version identifier corresponding to the target object after the target object is changed according to the current change operation; when When the target operation type does not match the delete operation type, it is determined that a conflict exists in the conflict check result corresponding to the object interaction request.
  • the blocking operation information writing stage is a stage of writing the blocking operation information into the storage space, and may also be called a write fence (write barrier) stage.
  • the blocking operation information may be information for blocking operations that do not meet the conditions.
  • the unqualified operation may be an operation whose version identifier is inconsistent with the current object version identifier corresponding to the target object.
  • the delete operation type is an operation type that can perform a delete operation, specifically, the delete operation type may be an operation type in a delete-only state. Further, only when the target operation type corresponds to a delete operation, the conflict checking result is determined as no conflict; otherwise, the conflict checking result is determined as conflicting.
  • the Schema will carry a version number, and each time an interactive operation is performed, the version number of the Schema can be stored as data in the storage node. In the write fence phase, the version number can be +1 operated.
  • the DDL statement for a certain Schema if the DDL statement for a certain Schema is executed, other nodes can know that the DDL statement for the Schema has been executed based on the broadcasted global object version, so the latest version number of the Schema can be obtained through ddl_job and latest status.
  • the DML statement accesses the Schema with a version number, if the version number is smaller than the current version number of the Schema, it means that the Schema is in the process of executing the DDL statement and has not completed it, and the DML statement needs to be rolled back. In this way, it can effectively prevent other statements from interacting with the Schema during the execution of DDL statements on the Schema.
  • the conflict checking result is determined based on the matching judgment of the operation type, which can effectively ensure the reliability of the conflict checking result.
  • the change stages corresponding to the current change operation may be shown as 502/504/506/508 and 510 in FIG. 5 .
  • Each change stage in Figure 5 can have its own status.
  • the server After receiving the object interaction request, the server will make a matching judgment between the target operation type corresponding to the object interaction request and the current parallel operation type in the current change stage. If both If they match, the conflict checking result is determined as no conflict, and if the two do not match, the conflict checking result is determined as conflicting.
  • the management node can also be called a cluster management node (MC, management of cluster), which can be realized by computer equipment to manage the information necessary for the entire cluster.
  • MC cluster management node
  • the server When receiving a DDL statement, the server reads the data dictionary from the storage node, obtains meta-information from it, and verifies the meta-information. If the meta-information verification fails, the execution of the response object interaction request (ABORT) is terminated.
  • ABST response object interaction request
  • the server determines the current state and target state of the Schema, obtains the state transition matrix from the local storage space, and determines whether to allow the current state to be converted to the target state based on the state transition matrix. If conversion is not allowed, execution is terminated in response to an object interaction request (ABORT).
  • ABSORT object interaction request
  • a preparation operation may be performed before the target change operation on the target object, and a finishing operation may be performed after the target change operation on the target object.
  • the preparation operation may be called start_ddl_jod
  • the finishing operation may be called end_ddl_jod
  • the target change operation may be called push_ddl_jod.
  • FIG. 6 is a schematic flowchart of an object processing method in an embodiment.
  • the specific implementation process may be: execute the preparation operation, if the preparation operation is successfully executed, it is determined that the start is successful, and the execution process of the native DDL process is entered.
  • the first native DDL process may be a DDL process that needs to be executed between the preparation operation and the object change operation.
  • the object change operation starts to be executed, and if the object change operation is executed successfully, it is determined that the presumption is successful.
  • the second native DDL process may be a DDL process that needs to be executed between an object change operation and a finishing operation. At the end of the execution of the second native DDL process, the finishing operation is started.
  • any type of DDL will record ddl record at the beginning of execution, that is, record state information into ddl_job. The purpose of this is to make it clear that a certain DDL has been executed in the system.
  • ddl_job is the DDL field.
  • DROP TABLE in order to ensure that DML will not read/write wrong data during the concurrent process of DDL, it is necessary to continuously adjust the status of metadata and user data according to specific situations during the execution of DDL. For example, before DROP TABLE actually deletes user data, it should advance the metadata state to the delete-only state to prevent new transactions from reading deleted user data.
  • the process of advancing the DDL state may be interrupted by any abnormality, such as network disk abnormality, etc.
  • the failure of any node in the promotion process means that the DDL execution fails. This situation requires a background thread to recover.
  • the recovery statement is the ddl_job recorded in start_ddl_job.
  • the ddl_job is continuously updated as the DDL progresses.
  • ddl_job can reserve the effective site for the recovery thread.
  • the scene where ddl_job is decided to be deleted means that the DDL execution is successful, otherwise the background recovery thread is required for a second judgment.
  • push_ddl_job uses It is the implementer of segmented DDL execution, as shown in Figure 5. Its main functions include:
  • This process can be a regular modification operation of the data dictionary; in addition, it also includes checking the meta information.
  • the state transition matrix is used to detect whether the state transition of the object is logical, and only allowed states can be advanced.
  • the promotion version identification is used to save the promotion information in case of failure recovery.
  • the object change operation in this embodiment of the present application can implement asynchronous deletion of data, for example: when a computing node actually executes an insert operation for a schema, another computing node can delete a certain record in the schema in one step.
  • the DROP operation in the DDL statement can be completed instantly, the background thread guarantees data deletion, and the actual deletion action does not need to be executed on the same computing node as the DDL statement.
  • start_ddl_job is the beginning of SQL Engine executing a DDL statement, as shown in Figure 7, its main functions include:
  • the DDL synchronization structure that is, scan the ddl_job to make sure that the cluster is not executing any statement that conflicts with the current ddl. For example: Execute the ALTER TABLE or DROP TABLE statement to obtain the conflict detection result according to the statement execution result.
  • the storage layer is used for the first conflict detection, that is, the concurrency control between DDLs is realized. If it is determined that there is a conflict based on the scan result of ddl_job, the execution of the DDL statement is terminated (ABORT).
  • the status information of this ddl is stored in the status record table (ddl_job table) for concurrency control and fault recovery. If the storage in the ddl_job table fails, the execution of the DDL statement (ABORT) is terminated.
  • start_ddl_job is executed successfully (SUCCESS).
  • the specific implementation process of the finishing operation may be as shown in FIG. 8 .
  • end_ddl_job is used to end DDL execution and perform some aftermath work.
  • the specific implementation process is described as follows:
  • the server locally decides whether to remove the DDL record from the ddl_job table based on the execution of the DDL. Among them, the basis for whether to remove is: whether it is necessary to restore the thread to resume or continue to execute, and if it is necessary to restore the thread to resume or continue to execute, it will not be removed.
  • release lock is executed, that is, release LOCK is executed, and the various global locks applied in the DDL execution process are released through the management node. If the execution of releasing the lock fails, recovery is performed through a background thread.
  • release lock When the release lock is executed successfully, the memory is released, that is, release mem is executed to release the corresponding memory. If the execution of freeing memory fails, recovery is performed through a background thread.
  • each stage divided in Fig. 5, Fig. 6, Fig. 7 and Fig. 8 can be understood as a complete operation stage, and the change stages required by different operation types can be deleted in a targeted manner, for example: the deletion operation does not The data dictionary information needs to be modified additionally.
  • the execution status of the DDL statement can be acquired by querying the DDL synchronization structure.
  • DDL_SQL create database d1
  • SCHEMA_NAME indicates the name of the target object
  • TABLE_NAME indicates the name of a table in the target object
  • TABLE_VERSION indicates the version of the table
  • DDL_STATUS indicates the execution status of the DDL statement
  • START_TIMESTAMP indicates the execution time of the DDL statement Start time
  • LAST_TIMESTAMP indicates the end time of executing the DDL statement
  • DDL_SQL indicates the DDL statement being executed
  • INFO indicates the information queried: exec_addr indicates the machine address for DDL execution, and ip indicates the corresponding IP (Internet Protocol, Internet Interconnection Protocol) address, port indicates the corresponding network interface, recov_addr indicates the address of the machine where the DDL execution is completed, and can include status information such as normal or abnormal.
  • the present application also provides an application scenario, where the above object processing method is applied.
  • the application of this object processing method in this application scenario is as follows:
  • the embodiments of the present application can be applied to distributed database systems, multi-write database cluster systems with separate computing and storage, and the like.
  • the process of executing a data definition language statement (DDL statement) is shown in FIG. 9 .
  • the push_ddl_job stage is used to implement the following functions: 1. Push status to ensure the visibility of user data; 2. Record ddl push information to ensure recovery at any node.
  • the end_ddl_job stage is used to implement aftermath work such as information preservation. In the case of a normal end, it is determined whether to retain the corresponding status record based on whether the temporary table needs to be cleaned up.
  • user data deletion needs to be executed by a background thread, that is, it needs to be executed by a DDL thread (DDL thread).
  • a failure record can be generated, and the state before the execution failure can be restored by restoring the thread.
  • the execution failure may be a program execution failure, or a failure of the system such as a power failure.
  • the DDL thread returns to the original state and can process other tasks; when it is necessary to restore the thread processing, restore the corresponding state by restoring the thread, and the restoration thread can access ddl_job to get the failure The state of execution, and then restore the state. There is no need to worry about data problems caused by DDL failure. Once DDL execution fails, the background thread will recover the abnormal DDL to ensure data consistency.
  • a cluster-level variable will be updated.
  • other nodes in the cluster start a new transaction, they will perform subsequent logical judgments based on the stage of the current schema change to obtain the cluster level.
  • variable determine whether to enter the Schema change judgment process according to the value of the variable at the cluster level, and judge the following: whether the currently operating schema has a Schema change, what changes have occurred, what state the Schema is in at present, and whether this state is Allows the execution of the current operation, etc., after processing execution is complete.
  • the schema of the computing nodes is also updated to the latest state.
  • a logic table can be added to the data dictionary, which is the state record table in the foregoing embodiment.
  • the state in the logical table is synchronized to determine how the statement should be executed.
  • This logical table can be accessed by all computing nodes in the computing node cluster, so the state recorded in the logical table can be used as a cluster-level variable.
  • the logic table can be implemented through ddl_job, and the DDL execution status can be queried in the form of SQL statements in ddl_job, and the record of ddl_job can be used to determine whether the current DDL has been executed and which step it has reached.
  • the synchronization structure in the logical table and the digital dictionary can be different.
  • the persistent update of the Schema state in the Schema change process can be used to prevent each node in the cluster from being executed.
  • the problem of data inconsistency caused by using the old version of the Schema object for operation add a logic to judge whether the current version of the Schema object is compatible with the operation version when data is written to the storage layer to determine whether to allow data reading and writing, so as to ensure that data reading and writing will not introduce data consistency and integrity problems. Multiple conflict resolution mechanisms are supported at the same time.
  • DDL takes precedence, that is, DDL can always advance the execution state, and other DML statement transactions will be rolled back in certain states. Specifically, if the DDL statement needs to be executed during the execution of other DML statements, these DML statements are rolled back after the execution of the DDL statement is completed.
  • DDL statements can be executed on multiple nodes, that is, DDL can be executed on any computing node, and irrelevant DDL can be executed in parallel to achieve multiple writes.
  • DDL statements are executed in stages, and other nodes can perceive them in time according to different stages, and can decide whether to continue executing/rolling back transactions according to the current execution stage of DDL. Through the fine division of state, the rollback transactions can be reduced to a minimum. Through the non-locking mode, different computing nodes in the cluster can perceive that the DDL change statement is occurring in the current system. Compared with the strong synchronization implementation, it can minimize the impact on other computing nodes when the DDL statement is executed.
  • the background thread can immediately roll back or advance based on the execution information to ensure that the entire distributed database system can enter another consistent state and restore the ability to provide external services.
  • the present application further provides an object processing device, which can be used to execute the above object processing method.
  • an object processing device which can be used to execute the above object processing method.
  • the illustrated structure does not constitute a limitation on the device, and may include More or fewer components, or combinations of certain components, or different arrangements of components.
  • an object processing apparatus 1000 is provided.
  • the apparatus may adopt a software module or a hardware module, or a combination of the two becomes a part of a computer device.
  • the apparatus specifically includes: an interaction request The receiving module 1002, the change stage acquiring module 1004, the conflict checking module 1006 and the operation executing module 1008, wherein:
  • An interaction request receiving module 1002 configured to receive an object interaction request for a target object.
  • the change stage obtaining module 1004 is used to respond to the object interaction request and obtain the current change stage of the current change operation for the target object; the current change operation includes multiple operation steps, and the current change operation is based on the conflict operation type corresponding to each operation step Divided into multiple change stages, the conflict operation type is the type of conflict operation corresponding to the operation step.
  • the conflict verification module 1006 is configured to determine the target operation type of the conflict verification result corresponding to the object interaction request according to the target operation type corresponding to the object interaction request and the current related operation type corresponding to the current change stage, which is requested by the object interaction request The type of target interaction.
  • An operation executing module 1008 configured to execute a target interaction operation on the target object according to the object interaction request when the result of the conflict check is that there is no conflict.
  • the change stage acquisition module further includes: an operation step acquisition submodule, used to obtain the current operation step corresponding to the current change operation on the target object; a change stage determination submodule, used to determine the current operation step
  • the corresponding change stage is the current change stage where the current change operation for the target object is located, wherein the continuous operation steps of different conflicting operation types are divided into different change stages.
  • the object processing device further includes: a step sequence determination module, configured to determine the current operation step sequence corresponding to the current change operation, the current operation step sequence includes a plurality of operation steps arranged in operation order; the conflict type The determination module is used to determine the conflict operation type corresponding to each operation step in the current operation step sequence; the step sequence division module is used to divide the current operation step sequence according to the conflict operation type corresponding to the operation step to obtain the current change operation A plurality of corresponding change phases, wherein consecutive operation steps of the same conflicting operation type are divided into the same change phase.
  • a step sequence determination module configured to determine the current operation step sequence corresponding to the current change operation, the current operation step sequence includes a plurality of operation steps arranged in operation order
  • the conflict type The determination module is used to determine the conflict operation type corresponding to each operation step in the current operation step sequence
  • the step sequence division module is used to divide the current operation step sequence according to the conflict operation type corresponding to the operation step to obtain the current change operation A plurality of corresponding change phases, wherein consecutive operation
  • the current relevant operation type includes the current conflicting operation type
  • the conflict checking module includes: a first operation type determination submodule, configured to extract the operation type identifier from the object interaction request, and according to the extracted operation type identifier Obtain the target operation type corresponding to the object interaction request; the first conflict verification submodule is used to compare the target operation type with the current conflict operation type, and determine the conflict check when it is determined according to the comparison result that the current conflict operation type includes the target operation type The test results show that there is a conflict.
  • the current relevant operation type includes the current parallel operation type
  • the conflict checking module includes: a second operation type determination submodule, configured to extract the operation type identifier from the object interaction request, and according to the extracted operation type identifier Obtain the target operation type corresponding to the object interaction request; the second conflict checking submodule is used to compare the target operation type with the current parallel operation type, and determine the conflict when it is determined according to the comparison result that the current parallel operation type does not include the target operation type The result of the verification is that there is a conflict.
  • the object processing apparatus further includes: an operation stop module, configured to stop executing the target interaction operation on the target object according to the object interaction request when the result of the conflict check is that there is a conflict.
  • the operation execution module includes: a step sequence determination submodule, used to determine the target operation step sequence corresponding to the target operation type in response to the object interaction request when the conflict check result shows that there is no conflict;
  • the operation execution sub-module is used to execute the target interaction operation on the target object according to the sequence of target operation steps.
  • the interactive operation execution submodule includes: a first operation stage determining unit, configured to determine the first operation stage corresponding to the target object, and the first operation stage is the next operation stage corresponding to the object interaction request; the lock request The sending unit is configured to send an operation lock request for the target object when the phase priority corresponding to the first operation phase meets the priority execution condition; the operation lock request is used to trigger the computing node that is interacting with the target object plus corresponding The object lock is used to lock in the current operation on the target object until it is unlocked; the operation step execution unit is configured to execute the operation step corresponding to the first operation stage when it is determined that the lock response information for the operation lock request is received; The lock response information is the response information returned by the computing node after adding the object lock.
  • the object processing device further includes: an operation stage determination module, configured to determine a second operation stage corresponding to the target object, where the second operation stage is the next operation stage corresponding to the first operation stage; an object lock maintenance module , for maintaining the object lock requested by the operation lock request and executing the operation steps corresponding to the second operation stage when the stage priority corresponding to the second operation stage satisfies the priority execution condition.
  • an operation stage determination module configured to determine a second operation stage corresponding to the target object, where the second operation stage is the next operation stage corresponding to the first operation stage
  • an object lock maintenance module for maintaining the object lock requested by the operation lock request and executing the operation steps corresponding to the second operation stage when the stage priority corresponding to the second operation stage satisfies the priority execution condition.
  • the lock request sending submodule includes: a content level determination unit, configured to determine the operation content level of the operation content corresponding to the first operation stage; a lock request sending unit, configured to determine the operation content level when the operation content level is the object level , it is determined that the phase priority corresponding to the first operation phase satisfies the priority execution condition, and an operation lock request for the target object is sent.
  • the object processing device further includes: a target version identifier determination module, configured to determine the target object version identifier corresponding to the object interaction request; After the change, the current object version identifier corresponding to the target object; the version identifier comparison module is used to compare the target object version identifier with the current object version identifier, and stop responding to the object interaction request when the comparison is inconsistent.
  • a target version identifier determination module configured to determine the target object version identifier corresponding to the object interaction request
  • the version identifier comparison module is used to compare the target object version identifier with the current object version identifier, and stop responding to the object interaction request when the comparison is inconsistent.
  • the current relevant operation type includes the current parallel operation type
  • the conflict check module includes: an operation type determination submodule, used to determine the current change phase when the current change phase is the blocking operation information writing phase
  • the current parallel operation type of is the delete operation type
  • the blocking operation information writing stage is used to write the current object version identifier corresponding to the target object after the target object is changed according to the current change operation
  • the third conflict checking submodule uses When the target operation type does not match the deletion operation type, it is determined that a conflict exists in the conflict checking result corresponding to the object interaction request.
  • Each module in the above-mentioned object processing apparatus may be fully or partially realized by software, hardware or a combination thereof.
  • the above-mentioned modules can be embedded in or independent of one or more processors in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that one or more processors can call and execute the above The operation corresponding to the module.
  • a computer device is provided.
  • the computer device may be a server, and its internal structure may be as shown in FIG. 11 .
  • the computer device includes one or more processors, memory, and network interfaces connected by a system bus. Wherein, one or more processors of the computer device are used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system, computer readable instructions and a database.
  • the internal memory provides an environment for the execution of the operating system and computer readable instructions in the non-volatile storage medium.
  • the database of the computer equipment is used to store data such as conflict checking results.
  • the network interface of the computer device is used to communicate with an external terminal via a network connection.
  • the computer readable instructions implement an object processing method when executed by one or more processors.
  • FIG. 11 is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation on the computer equipment on which the solution of this application is applied.
  • the specific computer equipment can be More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.
  • a computer device including a memory and one or more processors, where computer-readable instructions are stored in the memory, and the above-mentioned methods are implemented when the one or more processors execute the computer-readable instructions Steps in the examples.
  • one or more computer-readable storage media are provided, storing computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the steps in the foregoing method embodiments are implemented.
  • a computer program product comprising computer readable instructions stored in a computer readable storage medium.
  • One or more processors of the computer device read the computer-readable instructions from the computer-readable storage medium, and one or more processors execute the computer-readable instructions, so that the computer device executes the steps in the foregoing method embodiments.
  • Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory or optical memory, etc.
  • Volatile memory can include Random Access Memory (RAM) or external cache memory.
  • RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种对象处理方法,包括:响应于对象交互请求,获取针对目标对象的当前变更操作所在的当前变更阶段;当前变更操作根据各个操作步骤所对应的冲突操作类型划分为多个变更阶段;根据对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型,确定对象交互请求对应的冲突校验结果;目标操作类型,是对象交互请求所请求的目标交互操作的类型;当冲突校验结果为不存在冲突时,根据对象交互请求对目标对象执行目标交互操作。

Description

对象处理方法、装置、计算机设备和存储介质
本申请要求于2021年05月19日提交中国专利局,申请号为2021105475897、发明名称为“对象处理方法、装置、计算机设备和存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请涉及网络技术领域,特别是涉及一种对象处理方法、装置、计算机设备和存储介质。
背景技术
在网络技术领域,往往需要与数据库等中记录的对象进行交互。例如:在某些时候,需要对数据库中的对象进行变更操作。
传统技术中,是通过强同步的方式实现与对象的交互,即,允许特定的计算节点来集中执行交互语句。这就会导致其他计算节点需要等待集中控制的计算节点的执行结果才能进行后续的处理。导致对象交互处理的效率低下。
需要说明的是,在上述背景技术部分公开的信息仅用于加强对本申请的背景的理解,因此可以包括不构成对本领域普通技术人员已知的现有技术的信息。
发明内容
基于此,有必要针对上述技术问题,提供一种对象处理方法、装置、计算机设备和存储介质。
一种对象处理方法,由计算机设备执行,所述方法包括:接收针对目标对象的对象交互请求;响应于所述对象交互请求,获取针对所述目标对象的当前变更操作所在的当前变更阶段;所述当前变更操作包括多个操作步骤,所述当前变更操作根据各个所述操作步骤所对应的冲突操作类型划分为多个变更阶段,所述冲突操作类型为所述操作步骤所对应的冲突操作的类型;根据所述对象交互请求所对应的目标操作类型以及所述当前变更阶段对应的当前相关操作类型,确定所述对象交互请求对应的冲突校验结果;所述目标操作类型,是所述对象交互请求所请求的目标交互操作的类型;及当所述冲突校验结果为不存在冲突时,根据所述对象交互请求对所述目标对象执行所述目标交互操作。
一种对象处理装置,由计算机设备执行,所述装置包括:交互请求接收模块,用于接收针对目标对象的对象交互请求;变更阶段获取模块,用于响应于所述对象交互请求,获取针对所述目标对象的当前变更操作所在的当前变更阶段;所述当前变更操作包括多个操作步骤,所述当前变更操作根据各个所述操作步骤所对应的冲突操作类型划分为多个变更阶段,所述冲突操作类型为所述操作步骤所对应的冲突操作的类型;冲突校验模块,用于根据所述对象交互请求所对应的目标操作类型以及所述当前变更阶段对应的当前相关操作类型,确定所述对象交互请求对应的冲突校验结果;所述目标操作类型,是所述对象交互请求所请求的目标交互操作的类型;及操作执行模块,用于当所述冲突校验结果为不存在冲突时,根据所述对象交互请求对所述目标对象执行所述目标交互操作。
一种计算机设备,包括存储器和一个或多个处理器,所述存储器中存储有计算机可读指令,所述计算机可读指令被所述一个或多个处理器执行时,使得所述一个或多个处理器执行本申请各实施例所述的方法中的步骤。
一个或多个计算机可读存储介质,所述计算机可读存储介质上存储有计算机可读指令,所述计算机可读指令被一个或多个处理器执行时,使得所述一个或多个处理器执行本申请各实施例所述的方法中的步骤。
一种计算机程序产品,所述计算机程序产品包括计算机可读指令,所述计算机可读指令存储在计算机可读存储介质中;计算机设备的一个或多个处理器从计算机可读存储介质读取该计算机可读指令,一个或多个处理器执行该计算机可读指令,使得所述计算机设备执行本申请各实施例所述的方法中的步骤。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图:
图1为一个实施例中对象处理方法的应用环境图;
图2为另一个实施例中对象处理方法的应用环境图;
图3为一个实施例中对象处理方法的流程示意图;
图4为一个实施例中状态转换矩阵的示意图;
图5为一个实施例中变更操作的流程示意图;
图6为另一个实施例中对象处理方法的流程示意图;
图7为一个实施例中准备操作的流程示意图;
图8为一个实施例中收尾操作的流程示意图;
图9为一个实施例中执行数据定义语言语句的流程示意图;
图10为一个实施例中对象处理装置的结构框图;
图11为一个实施例中计算机设备的内部结构图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本申请进行进一步详细说明。应当理解,此处描述的具体实施例仅仅用以解释本申请,并不用于限定本申请。
本申请提供的对象处理方法,可以应用于如图1所示的应用环境中。该应用环境中包含服务器102、第一终端104、第二终端106以及数据库108。其中,第一终端104和第二终端106分别与服务器102通过网络进行通信,数据库108为用于存储各个终端数据的数据库,能够与服务器以及各个终端进行网络通信。在应用过程中,第一终端104向数据库108中的目标对象执行当前变更操作,当前变更操作对应有多个变更阶段,第一终端104在执行各个变更阶段时将每个变更阶段的执行状态发送至服务器。当接收到第二终端106发送的针对目标对象的对象交互请求时,服务器102通过第一终端104发送的各个变更阶段的执行状态获取第一终端104的当前变更操作所对应的当前变更阶段,根据对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型,确定对象交互请求对应的冲突校验结果,当冲突校验结果为存在冲突时,停止根据对象交互请求执行目标交互操作,当冲突校验结果为不存在冲突时,根据对象交互请求对目标对象执行目标交互操作,并将冲突校验结果发送给第二终端106。其中,服务器可以是独立的物理服务器,也可以是多个物理服务器构成的服务器集群或者分布式系统,还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、CDN(Content Delivery Network,即内容分发网络)、以及大数据和人工智能平台等基础云计算服务的云服务器。终端(包括第一终端和第二终端)可以是智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表等,但并不局限于此。终端以及服务器可以通过有线或无线通信方式进行直接或间接地连接,本申请在此不做限制。在其他应用场景中,第一终端104和第二终端106也可以通过一个终端设备实现,另外,终端也可以替换为服务器。在一些实施例中,对目标对象进行交互的终端,可以通过计算引擎节点(SQL Engine,也可以称为计算节点或者节点)实现,主要完成SQL(Structured Query Language,结构化查询语言)层计算功能。需要说明的是,本申请实施例中的多个可以为至少两个。
本申请提供的对象处理方法,还可以应用于如图2所示的应用环境中。该应用环境中包含终端202、计算节点集群204、以及数据库206。计算节点集群204可以包括至少一个计算机设备,计算机设备可以称为计算节点,这些计算节点可以通过服务器或中断实现。另外,计算节点集群204中的各个计算节点均可以对同一对象具有交互权限。图2中示出了第一计算节点2042和第二计算节点2044这两个计算节点。在实际的应用场景中,可以包括数量更少或者更多的计算节点。其中,第一计算节点2042和第二计算节点2044分别与数据库206通过网络进行通信,数据库206为用于存储各个计算节点数据的数据库。在应用过程中,第一计算节点2042向数据库206中的目标对象执行当前变更操作,当前变更操作对应有多个变更阶段,第一计算节点2042在执行各个变更阶段时将每个变更阶段的执行状态保 存至数据库206中。当第二计算节点2044接收到终端202发送的针对目标对象的对象交互请求时,从数据库中获取第一计算节点2042的当前变更操作下的各个变更阶段的执行状态,基于执行状态确定获取第一计算节点2042的当前变更操作所对应的当前变更阶段,根据对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型,确定对象交互请求对应的冲突校验结果,其中,目标操作类型,是对象交互请求所请求的目标交互操作的类型;当冲突校验结果为存在冲突时,停止根据对象交互请求执行目标交互操作,当冲突校验结果为不存在冲突时,根据对象交互请求对目标对象执行目标交互操作。
本申请提供的各个实施例可以基于云技术(Cloud technology)实现。其中,云技术是指在广域网或局域网内将硬件、软件、网络等系列资源统一起来,实现数据的计算、储存、处理和共享的一种托管技术。云技术基于云计算商业模式应用的网络技术、信息技术、整合技术、管理平台技术、应用技术等的总称,可以组成资源池,按需所用,灵活便利。云计算技术将变成重要支撑。技术网络系统的后台服务需要大量的计算、存储资源,如视频网站、图片类网站和更多的门户网站。伴随着互联网行业的高度发展和应用,将来每个物品都有可能存在自己的识别标志,都需要传输到后台系统进行逻辑处理,不同程度级别的数据将会分开处理,各类行业数据皆需要强大的系统后盾支撑,只能通过云计算来实现。
后台服务所需的存储资源可以通过数据库实现。其中,数据库(Database),简而言之可视为电子化的文件柜——存储电子文件的处所,用户可以对文件中的数据进行新增、查询、更新、删除等操作。所谓“数据库”是以一定方式储存在一起、能与多个用户共享、具有尽可能小的冗余度、与应用程序彼此独立的数据集合。数据库管理系统(英语:Database Management System,简称DBMS)是为管理数据库而设计的电脑软件系统,一般具有存储、截取、安全保障、备份等基础功能。数据库管理系统可以依据它所支持的数据库模型来作分类,例如关系式、XML(Extensible Markup Language,即可扩展标记语言);或依据所支持的计算机类型来作分类,例如服务器群集、移动电话;或依据所用查询语言来作分类,例如:SQL、XQuery;或依据性能冲量重点来作分类,例如最大规模、最高运行速度;亦或其他的分类方式。不论使用哪种分类方式,一些DBMS能够跨类别,例如,同时支持多种查询语言。
如本申请所提供的对象处理方法、装置、计算机设备和存储介质,在各实施例运行过程中生成的数据可以保存于区块链上。例如:目标对象的当前变更操作所在的当前变更阶段记录在区块链中;在接收到对象交互请求时,终端或服务器从区块链中获取当前变更阶段,确定当前变更阶段对应的当前相关操作类型,根据目标操作类型和当前相关操作类型确定对象交互请求对应的冲突校验结果;当确定存在冲突时,停止根据对象交互请求执行目标交互操作;当确定不存在冲突时继续根据对象交互请求执行目标交互操作,并将交互操作过程中各个阶段的状态信息记录到区块链中,另外,还可以把冲突校验结果记录到区块链中。终端或服务器可以从区块链中获取交互操作过程中各个阶段的状态信息。同时,在运行过程中生成的目标操作类型、当前相关操作类型和冲突校验结果等均可以记录在区块链中。通过将数据记录在区块链中,可以防止这些数据被篡改,保证这些数据的存储可靠性,进而保证对象处理过程的可靠性。
在一个实施例中,如图3所示,提供了一种对象处理方法,应用于计算机设备,可以理解,计算机设备可以是服务器,也可以是终端。本实施例以该方法应用于服务器进行举例说明,可以理解的是,该方法还可以应用于包括终端和服务器的系统,并通过终端和服务器的交互实现。本实施例中,该方法包括以下步骤:
S302,接收针对目标对象的对象交互请求。
其中,对象为网络交互所针对的主体,可以是网络设备、网络设备中的数据、数据库中的数据等中的至少一项。在某些实施例中,目标对象可以是数据库中存储的数据或者元数据等中的至少一项。目标对象具体可以是数据库中的Schema(元数据的一个抽象集合),例如:数据库中的以下至少一项:表、列、索引、约束、存储过程或者函数等等。进一步地,目标对象的数量可以为至少一个。当目标对象为至少两个时,可以分别为各个目标对象进行冲突校验,进而针对性地停止或者继续根据对象交 互请求对各个目标对象执行目标交互操作。
交互可以是对目标对象进行变更或者查询等操作中的至少一种。其中,变更可以是:新建、修改、删除、插入或者更新等中的至少一种。具体地,向表中插入一列可以认为是一次变更操作,插入两列可以认为是两次变更操作。
在一个实施例中,对象交互请求为对目标对象进行特定交互的请求。进一步地,各个对象交互请求可以对应至少一个对象交互操作,在确定对象交互请求后可以确定对应的对象交互操作,得到目标交互操作。一个对象交互请求中可以对应至少一个目标交互操作,例如,对于以下的对象交互请求:alter table t add a int,add b int,对应的目标交互操作为向表t中增加a和b两列,每个增加一列的操作可以认为是一个目标交互操作。当交互为变更操作时,如果接收到对象交互请求,服务器可以确定与对象交互请求对应的目标变更操作。
以目标对象为数据库中的Schema为例,对象交互操作可以为DML(Data Manipulation Language,数据操纵语言)、DDL(Data Definition Language,数据定义语言)或者DCL(Data Control Language,数据控制语言)等中的至少一项。其中,DML:可以是select、update、insert以及delete等命令,这4条命令是用来对数据库里的数据进行操作的语言;DDL:DDL的命令类型比DML要多,主要的命令有create、alter或者drop等,DDL主要是用在定义或改变表(table)的结构、数据类型、表之间的链接和约束等初始化工作上,大多在建立表时使用;DCL:是数据库控制功能,是用来设置或更改数据库用户或角色权限的语句,包括grant、deny或者revoke等语句。进一步地,DML、DDL或者DCL等语言是对数据库中的对象进行变更的操作,因此,可以称为对象变更操作,对应的请求可以称为对象变更请求。另外,对象交互操作还可以进行细分,例如:alter还可以细分为:copy(复制)、inplace(替换)或者online(在线)等操作,这些操作也可以基于操作步骤划分为多个变更阶段。进一步地,对象交互请求可以是服务器接收到的终端发送的命令语句,例如,服务器接收到终端发送的DDL语句,该DDL语句为对象交互请求。
在一个实施例中,服务器可以接收终端发送的针对目标对象的对象交互请求。
S304,响应于对象交互请求,获取针对目标对象的当前变更操作所在的当前变更阶段;当前变更操作包括多个操作步骤,当前变更操作根据各个操作步骤所对应的冲突操作类型划分为多个变更阶段,冲突操作类型为操作步骤所对应的冲突操作的类型。
其中,变更操作为对目标对象进行变更的操作,可以是对目标对象的内容、结构等进行变更的操作,具体可以是新建、修改、删除、插入或者更新等中的至少一种。当前变更操作可以为接收到对象交互请求时,特定计算节点正在对目标对象执行的变更操作,也可以是在接收到对象交互请求设定时间段之前或者设定时间段之后特定计算节点对目标对象执行的变更操作。变更操作包括多个变更阶段。其中,变更阶段为执行某一变更操作所细分的执行阶段。各个变更阶段可以包含至少一个操作步骤,同一变更阶段内的各个操作步骤对应的冲突操作类型相同,即,同一变更阶段在执行过程中如果出现计算节点C发出对象交互请求时,服务器做出的响应是唯一的。例如,触发计算节点C进行回滚或者继续执行。在一个实施例中,当前变更操作的执行优先级可以高于或等于对象交互请求对应的目标操作,进一步的,当前变更操作的各个变更阶段的执行优先级也可以高于或等于对象交互请求对应的各个交互阶段。进一步的,当前变更阶段可以为接收到对象交互请求时,特定计算节点正在对目标对象执行的变更阶段,也可以是在接收到对象交互请求设定时间段之前或者设定时间段之后特定计算节点对目标对象执行的变更阶段。
其中,一个操作可以包括多个操作步骤,即一个操作划分为多个操作步骤执行,通过该操作对应的多个操作步骤的执行,能够完成该操作所要达到的功能。操作步骤为操作对应的操作单元,例如一个操作可以包括建立握手连接、接口访问、获取元信息、校验元信息、状态标识更新、读取某个数据记录、数据记录对比或者数据记录修改等中的至少一种步骤。这些操作步骤按照一定的时序执行,以完成对应的变更操作。进一步的,操作步骤所对应的冲突操作为与操作步骤存在冲突的交互操作,即如果某一操作步骤与冲突操作并行运行,则可能导致目标对象对应的信息出现信息混乱,因此,服务器应当防止操作步骤与对应的冲突操作并行运行。进一步的,冲突操作类型为操作步骤所对应的冲突 操作的类型。其中,冲突操作可以包括多种类型,可以为新建、修改、删除、插入、更新或者查询等中的至少一种。
在一个实施例中,针对目标对象的当前变更操作可以是目标计算节点对目标对象进行的变更操作。其中,目标计算节点可以为区别于前述服务器的其他计算机设备。
在一个实施例中,对当前变更操作进行变更阶段划分的过程可以是:1、服务器按照一定的方式自动划分的,即,服务器自动根据冲突操作类型将当前变更操作划分为多个变更阶段;2、也可以是预先由人工划分的。
进一步地,对于服务器自动对变更操作划分变更阶段的情况,服务器可以确定各个操作步骤与计算节点集群中其他计算节点是否可以并行运行,当不能并行运行时,确定存在冲突,确定会产生冲突的计算节点对应的操作类型,作为对应操作步骤的冲突操作类型。更进一步地,可以将相同冲突操作类型下的连续的操作步骤划分到同一变更阶段中,还可以将不同冲突操作类型下的操作步骤划分到不同变更阶段中。其中,计算节点集群为计算节点构成的集群,这些计算节点能够与目标对象进行交互。进一步的,目标对象为目标数据库中的对象,目标数据库为服务器所对应的数据库,将能够访问该数据库的终端和服务器确定为计算节点,这些计算节点构成计算节点集群。
在一个实施例中,服务器可以设置状态记录表,状态记录表可以表示为ddl_job或者ddl_jobs。该状态记录表记录有针对目标对象的各个变更操作中各个变更阶段的执行状态,各个变更阶段对应的初始状态可以为未执行,计算节点每执行完一个变更阶段可以将状态从未执行修改为已执行。服务器可以在接收到对象交互请求时,访问状态记录表,以定位到当前变更操作对应的目标状态记录,按照变更阶段的顺序,将目标状态记录中第一个未执行的变更阶段确定为当前变更阶段。进一步的,当当前变更操作的某个变更阶段执行失败时,可以从状态记录表中获取最新执行完成的变更阶段的状态记录,进而基于该状态记录恢复到执行失败前的变更阶段,此时可以重新对执行失败的变更阶段进行执行。其中,状态记录表可以存储在服务器中。也可以存储在执行当前变更操作的计算节点中,还可以存储在服务器以及各个计算节点都可以访问的数据库中。
S306,根据对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型,确定对象交互请求对应的冲突校验结果,目标操作类型,是对象交互请求所请求的目标交互操作的类型。
在一个实施例中,当前相关操作类型包括当前冲突操作类型或者当前并行操作类型。
在一个实施例中,目标操作类型为目标交互操作的类型,可以为新建、修改、删除、插入、更新或者查询等中的至少一种。对象交互请求对应的目标操作类型可以为至少一种,当目标操作类型为至少两种时,可以分别将各个目标操作类型与当前变更阶段对应的当前相关操作类型进行对比,进而得到冲突校验结果。
在一个实施例中,当前冲突操作类型为不允许与当前变更阶段同步执行的操作类型,当前并行操作类型为允许与当前变更阶段同步执行的操作类型,当前冲突操作类型可以包含至少一个冲突操作类型,当前并行操作类型也可以包含至少一个并行操作类型。进一步地,可以将目标操作类型与当前相关操作类型进行匹配性判断,根据匹配性判断结果得到对象交互请求对应的冲突校验结果。具体地,如果当前冲突操作类型包含目标操作类型,则将冲突校验结果确定为存在冲突,如果当前并行操作类型包含目标操作类型时,则将冲突校验结果确定为不存在冲突。在某些实施例中,也可以将当前冲突操作类型和当前并行操作类型一起确定为当前相关操作类型。
在一个实施例中,当前冲突操作类型和当前并行操作类型可以根据预先确定的各个变更操作对应的操作类型记录表。进一步的,操作类型记录表可以包括冲突操作类型记录表和并行操作类型记录表。更进一步地,服务器在确定当前变更操作时,确定当前变更操作所对应的冲突操作类型记录表,将冲突操作类型记录表中记录的冲突操作类型确定为当前冲突操作类型。同样的,当前并行操作类型可以根据预先确定的各个变更操作的并行操作类型记录表确定。服务器在确定当前变更操作时,确定当前变更操作所对应的并行操作类型记录表,将并行操作类型记录表中记录的并行操作类型确定为当前并行操作类型。
S308,当冲突校验结果为不存在冲突时,根据对象交互请求对目标对象执行目标交互操作。
具体地,当对象交互请求为对象变更请求时,则对目标对象执行对应的目标变更操作。例如:当接收到DDL语句时,对目标对象执行该DDL语句对应的命令。
当不存在冲突时,说明允许对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型并行运行,可以根据对象交互请求执行目标交互操作。
在一个实施例中,当冲突校验结果为存在冲突时,停止根据对象交互请求执行目标交互操作。当存在冲突时,说明不允许对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型并行运行,可以停止根据对象交互请求执行目标交互操作。
上述对象处理方法中,通过将变更操作细分为变更阶段,能够在各个变更阶段下进行冲突校验,可以并行运行不存在冲突的操作阶段,同时可以停止运行存在冲突的操作阶段,可以保证操作的原子性,能在很大程度上降低变更操作对其他交互操作的影响,有效提高对象交互处理的效率。另外,将当前变更操作细分为变更阶段,细化了目标交互操作与当前变更操作之间的对比,目标交互操作不需要等到变更操作的所有阶段完全执行完,这种方式不会影响生产环境,能最大程度地降低变更操作对其他交互操作的影响,对于触发对象交互请求的用户来说是无感知的。
在一个实施例中,获取针对目标对象的当前变更操作所在的当前变更阶段包括:获取针对目标对象的当前变更操作所对应的当前操作步骤;确定当前操作步骤所对应的变更阶段,作为针对目标对象的当前变更操作所在的当前变更阶段,其中,不同的冲突操作类型的连续操作步骤划分在不同的变更阶段中。
其中,当前操作步骤可以为接收到对象交互请求时,特定计算节点正在对目标对象执行的操作步骤,也可以是在接收到对象交互请求设定时间段之前或者设定时间段之后特定计算节点对目标对象执行的操作步骤。
在一个实施例中,可以预先设置操作步骤与变更阶段的对应关系,也可以是即时确定操作步骤对应的变更阶段。服务器在接收到对象交互请求时,根据操作步骤与变更阶段的对应关系确定与当前操作步骤对应的当前变更阶段。
上述实施例中,基于当前变更操作所对应的当前操作步骤确定当前变更阶段,进而基于当前变更阶段对对象交互请求进行冲突校验。能通过细化的操作步骤确定出准确的当前变更阶段,进而得到可靠的冲突校验结果。
在一个实施例中,对当前变更操作进行阶段划分的步骤包括:确定当前变更操作所对应的当前操作步骤序列,当前操作步骤序列包括多个按照操作顺序进行排列的操作步骤;确定当前操作步骤序列中各个操作步骤所对应的冲突操作类型;根据操作步骤所对应的冲突操作类型对当前操作步骤序列进行划分,得到当前变更操作所对应的多个变更阶段,其中,相同冲突操作类型的连续操作步骤划分在相同的变更阶段中。
其中,操作步骤序列为多个操作步骤按照执行顺序排序得到的序列。执行顺序相邻的至少两个操作步骤可以认为是连续的操作步骤。即对于相邻的操作步骤,如果对应的冲突操作类型不同,则划分在不同的变更阶段中。
在一个实施例中,变更操作包括多个操作步骤。以新建操作为例,新建操作可以包括以下多个连续的操作步骤:1、获取元信息,2、校验元信息,3、向数据字典中添加新表数据,其中,数据字典可以存储在数据库中,4、状态标识更新等。这些操作步骤构成当前操作步骤序列。各个操作步骤在执行过程中与其他计算节点的交互操作可能存在不同的冲突状态。例如,假设另一操作为插入操作,插入操作的各个操作步骤可以与新建操作中的获取元信息、校验元信息、状态标识更新的操作步骤并行运行,即不存在冲突,而插入操作的各个操作步骤与新建操作中向数据字典中添加新表数据的操作步骤无法并行运行,需插入操作回滚,即存在冲突。进一步的,相互连续的操作步骤1和2对应的冲突操作类型相同,因此,可以将这两个操作步骤划分到同一变更阶段中,作为元信息校验阶段,而操作步骤2和3对应的冲突操作类型不同,因此,可以将这两个操作步骤划分到不同的变更阶段中,同时,操作步骤3和4对应的冲突操作类型不同,因此,可以将这两个操作步骤划分到不同的变更阶段中,分别将操作步骤3和4划分到数据添加阶段和标识更新阶段中。据此,新建操作对应的操作步骤序列 被划分为元信息校验阶段、数据添加阶段和标识更新阶段。在实际进行变更操作划分的过程中,所比较的操作的数量可以更多,也可以为计算节点集群中所有对同一对象进行的交互操作。
上述实施例,根据操作步骤序列中各个操作步骤所对应的冲突操作类型来对变更操作进行变更阶段的划分,以将相同冲突操作类型的连续操作步骤划分在相同的变更阶段中,将不同冲突操作类型的连续操作步骤划分在不同的变更阶段中,能基于操作步骤所对应的冲突操作类型准确实现变更阶段的划分。
在一个实施例中,根据对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型,确定对象交互请求对应的冲突校验结果包括:从对象交互请求中提取操作类型标识,根据所提取的操作类型标识得到对象交互请求对应的目标操作类型;将目标操作类型与当前冲突操作类型进行对比,当根据对比结果确定当前冲突操作类型包括目标操作类型时,确定冲突校验结果为存在冲突。
其中,计算节点在发送对象交互请求时可以携带目标操作类型的类型标识。服务器从对象交互请求携带的信息中提取类型标识即可以得到目标操作类型。
在一个实施例中,如果当前冲突操作类型为多个,可以将目标操作类型逐个与当前冲突操作类型进行对比。当存在与目标操作类型一致的冲突操作类型时,判定当前冲突操作类型包括目标操作类型,此时,将确定冲突校验结果为存在冲突;当不存在与目标操作类型一致的冲突操作类型时,判定当前冲突操作类型不包括目标操作类型,此时,将确定冲突校验结果为不存在冲突。
上述实施例,基于目标操作类型与当前冲突操作类型的对比结果进行冲突校验,进而可以快速得到冲突校验结果,有效提高对象处理的效率。
在一个实施例中,根据对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型,确定对象交互请求对应的冲突校验结果包括:从对象交互请求中提取操作类型标识,根据所提取的操作类型标识得到对象交互请求对应的目标操作类型;将目标操作类型与当前并行操作类型进行对比,当根据对比结果确定当前并行操作类型不包括目标操作类型时,确定冲突校验结果为存在冲突。
在一个实施例中,如果当前并行操作类型为多个,可以将目标操作类型逐个与当前并行操作类型进行对比。当不存在与目标操作类型一致的冲突操作类型时,判定当前并行操作类型不包括目标操作类型,此时,将确定冲突校验结果为存在冲突;同样的,当存在与目标操作类型一致的冲突操作类型时,判定当前并行操作类型包括目标操作类型,此时,将确定冲突校验结果为不存在冲突。
上述实施例,基于目标操作类型与当前并行操作类型的对比结果进行冲突校验,进而可以快速得到冲突校验结果,有效提高对象处理的效率。
在一个实施例中,操作类型记录表可以通过状态转换矩阵来实现,该状态转换矩阵可以记录目标对象的各个状态之间的转换可能性,如果可以转换,则记录为允许转换,如果不可以转换,则记录为不允许转换。进一步地,可以根据状态转换矩阵确定对象交互请求对应的冲突校验结果。其中,目标对象的状态可以为目标对象的数据读写状态,可以为公开(public)、可读(read)、可写(write)、不可读(no write)、不可写(no read)、等待(waiting)或者仅删除(delete only)等中的至少一种状态。其中,公开指的是可以进行读、写以及删除操作的状态。
在一个实施例中,各个操作类型可以对应不同的目标对象的状态。在一个实施例中,可以根据目标操作类型确定目标对象的目标状态,根据当前变更操作的操作类型确定目标对象的当前状态。具体地,如果当前变更操作未执行完,则当前状态可以为执行当前变更操作之前的状态,如果当前变更操作执行完毕,则当前状态可以为执行当前变更操作之后的状态。另外,可以将目标操作类型对目标对象的预期状态确定为目标对象的目标状态。
在一个实施例中,各个变更阶段也可以对应有状态,也可以根据当前变更阶段的执行状态确定目标对象的当前状态,具体过程可以参见上述根据当前变更操作确定当前状态的过程,在此不再赘述。
在一个实施例中,确定根据目标操作类型对目标对象进行交互之后目标对象对应的目标状态,当基于状态转换矩阵确定允许将目标对象从当前状态转换为目标状态时,将冲突校验结果确定为不存在冲突;否则,将对象变更请求对应的冲突校验结果确定为存在冲突。
在一个实施例中,状态转换矩阵可以如图4所示。图4记录的是是否允许将列项的状态转换为行项的状态,其中,表格中的箭头表示的转换方向,Y表示允许,N表示不允许。以第一列的第一个状态转换为第一行的第二个状态为例,目标对象的当前状态为新建-公开,即,当前变更操作为新建操作,该新建操作在执行完毕时对应的状态为公开;目标对象的目标对象为修改-公开,即,目标交互操作为修改操作,该修改操作在执行完毕时对应的状态为公开。通过查询图4中的状态转换矩阵的信息确定允许进行这种转换。因此,将冲突校验结果确定为不存在冲突。
在一个实施例中,方法还包括:当冲突校验结果为存在冲突时,停止根据对象交互请求对目标对象执行目标交互操作。
当存在冲突时,说明不允许对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型并行运行,可以停止根据对象交互请求执行目标交互操作。
在一个实施例中,当存在冲突时,还可以执行回滚操作,以恢复到接收对象交互请求之前的状态。之后可以实时监测当前变更操作的执行状态,如果当前变更操作执行完毕,获取当前变更操作对目标对象进行变更后的最新状态,可以再次响应对象交互请求,进而基于目标对象的最新状态执行目标交互操作。
通过这样的方式,能使得交互过程是基于目标对象的最新状态进行的,防止不同计算节点所获取的目标对象的状态存在差异。
在一个实施例中,当冲突校验结果为不存在冲突时,根据对象交互请求对目标对象执行目标交互操作,包括:当冲突校验结果为不存在冲突时,响应于对象交互请求,确定目标操作类型所对应的目标操作步骤序列;按照目标操作步骤序列对目标对象执行目标交互操作。
在一个实施例中,可以通过步骤序列记录表记录各种操作类型对应的操作步骤序列。在接收到对象交互请求并确定冲突校验结果为不存在冲突时,基于目标操作类型的操作类型标识从步骤序列记录表确定与操作类型标识对应的操作步骤序列,得到目标操作步骤序列。
上述实施例,确定目标操作类型所对应的目标操作步骤序列,进而按照目标操作步骤序列对目标对象执行目标交互操作,能够对目标对象执行与对象交互请求目标交互操作,保证对对象交互请求的可靠响应。
在一个实施例中,按照目标操作步骤序列对目标对象执行目标交互操作包括:确定目标对象对应的第一操作阶段,第一操作阶段为对象交互请求对应的下一操作阶段;当第一操作阶段所对应的阶段优先级满足优先执行条件时,发送针对目标对象的操作锁定请求;当确定接收到针对操作锁定请求的锁定响应信息时,执行第一操作阶段所对应的操作步骤。
其中,操作锁定请求为触发进行操作锁定的请求。进一步地,向正在与目标对象进行交互的计算节点发送操作锁定请求,触发这些计算节点加上对应的对象锁,以锁定在对目标对象的当前操作中,直到解锁。锁定响应信息可以为计算节点在加上对象锁之后向服务器返回的响应信息,即锁申请成功的反馈。服务器在接收到锁定响应信息时,可以判定对应的计算节点加上了对应的对象锁。
在一个实施例中,确定第一操作阶段所对应的阶段优先级是否满足优先执行条件的过程可以为:获取对目标对象进行交互的各个交互操作中的各个阶段优先级,将第一操作阶段所对应的阶段优先级与前述各个阶段优先级进行对比,当第一操作阶段所对应的阶段优先级最高时,确定第一操作阶段所对应的阶段优先级满足优先执行条件。
在一个实施例中,发送针对目标对象的操作锁定请求的实现过程可以为:向计算节点集群中的其他计算节点发送针对目标对象的操作锁定请求,以使得这些计算节点向服务器返回锁定响应信息。进一步地,可以向计算节点集群中的所有计算节点发送操作锁定请求,此时可以认为是申请全局锁。在某些情况下,也可以向计算节点集群中的部分计算节点发送操作锁定请求,此时可以认为是申请局部锁。在一个实施例中,向计算节点集群中的其他计算节点发送针对目标对象的操作锁定请求之后,可以在接收到部分计算节点的锁定响应信息时执行第一操作阶段所对应的操作步骤;也可以在接收到所有计算节点的锁定响应信息时执行第一操作阶段所对应的操作步骤。
进一步地,按照目标操作步骤序列对目标对象执行目标交互操作包括:确定目标对象对应的下一 操作阶段,得到第一操作阶段;向计算节点集群发送操作锁定请求,以使得计算节点集群中的各个计算节点在与操作锁定请求相容时加对象锁并返回锁定响应信息;当接收到计算节点集群中的各个计算节点返回的锁定响应信息时,执行第一操作阶段;当第一操作阶段执行完毕时向计算节点集群发送操作解锁请求,以使得计算节点集群中的各个计算节点解除对象锁并继续运行;返回根据对象变更请求获取对目标对象进行变更的下一变更阶段的步骤,直到对象变更请求对应的各个操作变更阶段执行完毕。
在一个实施例中,将目标交互操作分为多个小操作阶段来执行,可以不对所有的操作阶段都申请锁,在需要进行控制的操作阶段开始时再申请锁,每个操作阶段执行完毕后释放锁即可。具体地,表A中的数据存放在数据表中,正在基于某一对象变更操作删除表A,表A已经被删除,当前正属于删除数据表中表A的数据的过程,如果此时接收到新建一个同名的表的变更操作,由于表A已经被删除了,新建的表与删除的表不会存在冲突,与删除数据表的操作也不会冲突,因此新建表的过程就不需要申请锁。
在一个实施例中,以目标交互操作为DDL语句为例,整个计算节点集群对外可以表现为一个整体,在DDL进入的下一个阶段可能会导致其他DML语句回滚之前,服务器需要申请全局锁,此时可以进行强同步。申请全局锁的具体过程可以为:
1、服务器先向计算节点集群所有节点发送申请锁请求,并等待一段时间;
2、服务器检查是否有其他计算节点反馈已经在持有不相容的锁,如果有则等待,进入3;如果没有持有锁的反馈或者接收到锁定响应信息,则说明可以推进到下一阶段,进入4;
3、服务器等待各个计算节点的锁定响应信息;
4、服务器推进目标对象的状态。由于状态的推进,其他计算节点如果在前面的阶段没有发送反馈,可能会据此发生回滚。
5、服务器执行完成后,向反馈的计算节点发送锁释放消息,继续循环进行上述步骤,直到完成DDL语句。
上述实施例中,服务器在各个操作阶段都请求操作锁定,能使得各个操作阶段作为独立的操作单元,进而可以实现最小单元的冲突校验,尽可能地减少计算节点的等待过程。
在一个实施例中,对象处理方法还包括:确定目标对象对应的第二操作阶段,第二操作阶段为第一操作阶段对应的下一操作阶段;当第二操作阶段所对应的阶段优先级满足优先执行条件时,维持操作锁定请求所请求的对象锁,执行第二操作阶段所对应的操作步骤。
上述实施例中,第一操作阶段申请加锁之后,如果第二操作阶段还是要申请锁,则不释放第一操作阶段所申请的锁,以防止重复进行锁申请操作,能有效提高对象处理的效率。
在一个实施例中,当第一操作阶段所对应的阶段优先级满足优先执行条件时,发送针对目标对象的操作锁定请求包括:确定第一操作阶段所对应的操作内容的操作内容级别;当操作内容级别为对象级别时,确定第一操作阶段所对应的阶段优先级满足优先执行条件,发送针对目标对象的操作锁定请求。
其中,操作内容为交互操作对目标对象所操作的内容,操作内容可以是对象级别的内容,也可以是数据级别的内容。以目标对象为表为例,操作内容可以是对象级别的内容,例如:对表的结构、数据类型、表之间的链接和约束等进行的操作,操作内容可以是数据级别的内容,例如:向表中插入一条记录、对表中的某条记录进行数据修改等。内容级别的交互操作对目标对象的影响程度大于数据级别的交互操作对目标对象的影响程度。
上述实施例,基于操作内容级别来确定操作阶段对应的阶段优先级,进而确定是否发送操作锁定请求。能优先执行内容级别的交互操作,而数据级别的交互操作靠后执行,这样能有效防止执行数据级别的交互操作后,内容级别的交互操作覆盖数据级别的交互操作的影响,出现无效的操作,通过这种方式可以有效提高对象处理的效率。
在一个实施例中,第一操作阶段所对应的阶段优先级也可以根据目标操作类型确定,当目标操作类型符合预设的类型条件,则确定目标交互操作的各个操作阶段对应的阶段优先级均满足优先执行条 件。进一步地,当目标交互操作的操作内容级别为对象级别,则判定目标操作类型符合预设的类型条件,并确定目标交互操作的各个操作阶段对应的阶段优先级均满足优先执行条件。具体地,当目标操作类型为DDL语句对应的操作类型时,判定确定目标交互操作的各个操作阶段对应的阶段优先级均满足优先执行条件,即,DDL语句的执行优先于高于DML等语句。
在一个实施例中,对象处理方法还包括:确定对象交互请求所对应的目标对象版本标识;获取根据当前变更操作对目标对象进行变更后,目标对象所对应的当前对象版本标识;将目标对象版本标识与当前对象版本标识进行对比,当对比不一致时,停止响应对象交互请求。
其中,对象交互请求中携带目标对象的版本标识,服务器在接收到对象交互请求之后,确定该请求所携带的目标对象的版本标识,得到目标对象版本标识。
其中,目标对象对应有版本标识,对目标对象进行更新之后,目标对象的版本标识会发生变化,例如:每对目标对象进行一次变更,则目标对象的版本标识+1,据此可以得到目标对象所对应的当前对象版本标识。
上述实施例中,通过对目标对象的版本标识的对比来确定是否响应对象交互请求,只有目标对象版本标识与当前对象版本标识进行对比一致时,才响应对象交互请求。能使得对目标对象进行的交互操作都是以目标对象的最新状态来进行的,防止不同计算节点所获取的目标对象的内容存在差别,可以保证计算节点集群中所有计算节点对于目标对象的数据一致性。
在一个实施例中,对象交互请求为对象变更请求,根据对象交互请求执行目标交互操作,包括:当下一对象变更阶段为信息校验阶段时,获取与对象交互请求对应的交互基础信息;对交互基础信息进行校验;当交互基础信息校验通过时,对目标对象进行状态变更。
在一个实施例中,对象交互请求为对象变更请求,根据对象交互请求执行目标交互操作,包括:当下一对象变更阶段为版本标识写入阶段时,对目标对象进行对应的进行状态变更操作;获取进行状态变更操作后目标对象对应的目标状态版本标识;将目标状态版本标识写入存储空间中,以使得存储空间中的存储节点在接收到针对目标对象的新的对象交互请求时,确定新的对象交互请求中携带的状态版本标识,并在对象交互请求中携带的状态版本标识与目标状态版本标识不匹配时,停止响应该新的对象交互请求。其中,存储节点也可以称为存储引擎节点(TD Store),可以为能够运行在存储层中的计算机设备,主要完成事务及存储功能。
在一个实施例中,对象交互请求为对象变更请求,根据对象交互请求执行目标交互操作,包括:当下一对象变更阶段为状态广播阶段时,对目标对象进行对应的进行状态变更操作;基于状态变更操作生成状态变更广播信息;状态变更广播信息用于触发计算节点集群中的各个计算节点获取状态变更操作后的目标对象。其中,状态变更广播信息为目标对象的状态发生变更时生成的广播信息。各个计算节点在接收到状态变更广播信息能够从状态记录表中获取目标对象对应的最新状态和最新版本信息。进一步的,状态变更广播新可以通过global schema version(全局对象版本标识,GLSV)等表示。
上述实施例中,通过不同的变更阶段来响应对象变更请求,这些变更阶段的执行先后顺序可以根据实际情况调整,另外,不同目标操作类型下的变更阶段的数量和顺序可以不同。通过这种方式,能使得对象变更请求的响应通过多个独立的变更阶段实现,保证变更操作的原子化控制。
在一个实施例中,根据对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型,确定对象交互请求对应的冲突校验结果,包括:从存储空间中获取状态记录表,状态记录表用于记录各个对象变更操作对应的状态信息;当状态记录表中不存在冲突对象变更操作的状态信息时,根据对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型,确定对象交互请求对应的冲突校验结果;冲突对象变更操作为与对象交互请求相冲突的变更操作。本实施例中进行的冲突校验可以认为是交互操作之间的冲突校验,可以认为是第一级的冲突校验。
在一个实施例中,对象交互请求为对象变更请求,当状态记录表中不存在冲突对象变更操作的状态信息时,根据对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型,确定对象交互请求对应的冲突校验结果,包括:获取目标对象的对象标识;基于对象标识向集群管理节点发送对象注册请求,以使得集群管理节点响应于对象注册请求,基于对象标识生成唯一性注册号; 当获取到集群管理节点发送的唯一性注册号时,根据对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型,确定对象交互请求对应的冲突校验结果。本实施例中进行的冲突校验可以认为是交互操作内的冲突校验,可以认为是第二级的冲突校验。
在一个实施例中,当前变更操作的每个变更阶段可以对应有操作类型,这些操作类型可以基于预先配置的目标对象的状态来确定,例如:对于delete-only状态下的操作阶段,操作类型可以为删除操作类型;对于write状态下的操作阶段,操作类型可以为写操作类型;对于read状态下的操作阶段,操作类型可以为读操作类型;等等。
在一个实施例中,根据对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型,确定对象交互请求对应的冲突校验结果包括:当当前变更阶段为阻塞操作信息写入阶段时,确定当前变更阶段所对应的当前并行操作类型为删除操作类型;阻塞操作信息写入阶段用于写入根据当前变更操作对目标对象进行变更后,目标对象所对应的当前对象版本标识;当目标操作类型与删除操作类型不匹配时,确定对象交互请求对应的冲突校验结果为存在冲突。
其中,阻塞操作信息写入阶段为将阻塞操作信息写入存储空间的阶段,也可以称为write fence(写屏障)阶段。阻塞操作信息可以是用于阻塞不符合条件的操作的信息。其中不符合条件的操作可以是版本标识与目标对象所对应的当前对象版本标识不一致的操作。
其中,删除操作类型为可以进行删除操作的操作类型,具体地,删除操作类型可以为delete-only状态下的操作类型。进一步地,只有目标操作类型对应为删除操作时,才将冲突校验结果确定为不存在冲突,否则将冲突校验结果确定为存在冲突。
在一个实施例中,Schema会带上一个版本号,每执行完一个交互操作,可以将Schema的版本号作为数据存储到存储节点中。在write fence阶段可以将版本号进行+1操作。
在一个实施例中,如果针对某个Schema的DDL语句执行完毕,则其他节点能够基于所广播的全局对象版本知晓针对Schema的DDL语句已执行完毕,因此可以通过ddl_job来获取Schema的最新版本号以及最新状态。当DML语句带着一个版本号去访问Schema时,如果这个版本号比Schema当前的版本号小的话,说明Schema正处于执行DDL语句的流程中且未完成,需要DML语句回滚。通过这种方式,能有效防止对Schema执行DDL语句的过程中,其他语句对Schema进行交互。
上述实施例,在阻塞操作信息写入阶段中,基于操作类型的匹配性判断来确定冲突校验结果,能有效保证冲突校验结果的可靠性。
在一个实施例中,当前变更操作对应的变更阶段可以如图5中的502/504/506/508以及510所示。图5中的各个变更阶段都可以对应有各自的状态,服务器在接收到对象交互请求之后,将对象交互请求对应的目标操作类型与当前变更阶段的当前并行操作类型进行匹配性判断,如果两者相匹配则将冲突校验结果确定为不存在冲突,如果两者不匹配则将冲突校验结果确定为存在冲突。管理节点也可以称为集群管理节点(MC,management of cluster),可以通过计算机设备实现,以管理整个集群必须的信息。
进一步地,对当前变更操作对应的变更阶段详细说明如下:
1、在接收到DDL语句时,服务器从存储节点中读数据字典,从中获取元信息,并对元信息进行校验。如果元信息校验失败,则终止执行响应对象交互请求(ABORT)。
2、如果元信息校验成功,则服务器确定Schema的当前状态和目标状态,从本地存储空间中获取状态转换矩阵,并基于状态转换矩阵确定是否允许将当前状态转换为目标状态。如果不允许转换,则终止执行响应对象交互请求(ABORT)。
3、如果允许转换,则向存储节点推进write fence,以将Schema的版本号作为数据存储到存储节点中。如果推进失败,则终止执行响应对象交互请求(ABORT)。
4、如果推进成功,则通过存储节点更新数据字典。如果更新失败,则终止执行响应对象交互请求(ABORT)。
5、如果更新成功,则向管理节点推进GLSV,以通知其他计算针对Schema的语句所执行的状态。
6、如果推进GLSV成功,则当前变更操作执行成功(SUCCESS)。
在一个实施例中,当对象交互请求为对象变更请求时,在对目标对象的目标变更操作之前还可以执行准备操作,在对目标对象的目标变更操作之后还可以执行收尾操作。以目标变更操作为DDL对应的操作为例,准备操作可以称为start_ddl_jod,收尾操作可以称为end_ddl_jod,而目标变更操作可以称为push_ddl_jod。准备操作、目标变更操作以及收尾操作一起的执行过程可以如图6所示,图6为一个实施例中对象处理方法的流程示意图。具体的实现过程可以为:执行准备操作,如果准备操作执行成功,则判定开始成功,进入原生DDL流程的执行过程。其中,第一原生DDL流程可以为从准备操作到对象变更操作之间需要执行的DDL流程。在第一原生DDL流程执行结束时,开始执行对象变更操作,如果对象变更操作执行成功,判定推定成功。继续执行第二原生DDL流程。其中,第二原生DDL流程可以为从对象变更操作到收尾操作之间需要执行的DDL流程。在第二原生DDL流程执行结束时,开始执行收尾操作,如果收尾操作执行成功,判定推定成功,可以结束整个流程。另外,如图6所示,在某个操作步骤执行失败时可以判断是否需要恢复,如果需要恢复则执行恢复流程以恢复到执行失败之前的状态,保证对象变更操作的稳定运行。
在一个实施例中,任何类型的DDL都会在开始执行的时候记录ddl record,也即,将状态信息记录到ddl_job中。这样做的目的在于明确系统中已经执行过某DDL。对于恢复线程来说,ddl_job是DDL的现场。
在一个实施例中,为了保证DML在与DDL并发的过程中不会读/写错数据,在DDL执行的流程中就需要根据具体情形不断调整元数据和用户数据的状态。比如说DROP TABLE在没有真正执行删除用户数据之前,应当将元数据状态推进至delete-only状态,以防止新事务读已被删除的用户数据。在分布式数据库中,推进DDL状态的过程可能被任何异常打断,如网络磁盘异常等。而推进过程的任一节点失败都意味着DDL执行失败。这种情形需要后台线程来进行恢复,恢复的语句就是在start_ddl_job记录的ddl_job,ddl_job是随着DDL推进状态不断更新的,ddl_job的实时性可以为恢复线程保留有效现场。当DDL执行完毕时,需要根据DDL执行主流程是否抛出异常决定保留或者删除ddl_job。决定删除ddl_job的场景,即表示DDL执行成功,否则需要后台恢复线程进行二次判断。
具体地,push ddl job这个功能封装了将Schema状态从一个状态推进到下一个状态的全部必须状态,通过这个功能的执行,将一个DDL变更的执行划分为几个连续的小阶段执行;push_ddl_job用来推进DDL语句小阶段执行的状态,是分段DDL执行的实施者,如图5所示,其主要功能包括:
1、基于最新的数据字典信息做出版本信息的修改,在数据字典中修改Schema的状态和版本,这个过程可以是数据字典常规的修改操作;另外,还包括对元信息进行校验。
2、元信息校验成功时,通过状态转换矩阵检测对象状态转换是否符合逻辑,只有被允许的状态之间才可以进行状态推进。
3、添加/推进write fence阶段。
4、更新数据字典。
5、推进版本标识,以保存推进信息,以备故障恢复。
在一个实施例中,本申请实施例的对象变更操作可以实现异步删除数据,例如:当某一计算节点真正执行针对Schema的插入操作时,另一计算节点可以一步删除Schema中的某一记录。对于DDL语句中的DROP类操作可以瞬间完成,后台线程保证数据删除,并且实际删除动作也不需要与DDL语句在同一个计算节点执行。
在一个实施例中,准备操作的具体实现流程可以如图7所示。start_ddl_job是SQL Engine执行一个DDL语句的开端,如图7所示,其主要功能包括:
1、查询DDL同步结构,即扫描ddl_job,确定集群全局没有在执行与当前ddl冲突的语句。例如:执行ALTER TABLE或者DROP TABLE语句,以根据语句执行结果得到冲突检测结果,这里是利用存储层来做第一次冲突检测,即实现DDL之间的并发控制。如果基于ddl_job的扫描结果确定存在冲突,则终止执行DDL语句(ABORT)。
2、如果基于ddl_job的扫描结果确定无冲突,获取目标对象的库表名形成的key(关键词),并在管理节点进行注册,管理节点在注册过程中不允许两个重复值的注册,MC在注册成功时会输出数据 字典锁(dd.table lock),这里是利用集群管理层来做的第二次冲突检测,即实现DDL语句间的并发控制。如果dd.table lock获取失败,即,注册失败,则终止执行DDL语句(ABORT)。
3、如果dd.table lock获取成功,将本次ddl的状态信息存入状态记录表(ddl_job表)中,以进行并发控制与故障恢复。如果存入ddl_job表失败,则终止执行DDL语句(ABORT)。
4、检查3中存入记录的存在性,当存在时,说明存入ddl_job表成功,从管理节点中获取状态锁(jod_id lock,job-id锁)以快速判断当前DDL的执行状态,如果获取不到则说明DDL尚在正常流程中执行或者已经在其他节点上恢复。如果job-id锁获取失败,则终止执行DDL语句(ABORT)。
5、如果job-id锁获取成功,则start_ddl_job执行成功(SUCCESS)。
在一个实施例中,收尾操作的具体实现流程可以如图8所示。具体地,end_ddl_job用来结束DDL执行,进行一些善后工作,具体的实现流程说明如下:
1、服务器本地根据DDL的执行现场决策是否需要从ddl_job表中移除DDL记录。其中,是否需要移除的依据是:是否需要恢复线程来恢复或者继续执行,如果需要恢复线程来恢复或者需要继续执行,则不移除。
2、当决策成功时,通过管理节点推进GLSV(即,推对象版本标识),以通知其他计算针对Schema的语句已经执行结束。
3、当推进GLSV成功时,执行释放锁,即执行release LOCK,通过管理节点释放本DDL执行流程中申请的各种全局锁。如果释放锁执行失败,则通过后台线程来进行恢复。
4、当释放锁执行成功时,执行释放内存,即执行release mem,释放相应的内存。如果释放内存执行失败,则通过后台线程来进行恢复。
5、如果释放内存执行成功,则end_ddl_job执行成功(SUCCESS)。另外,如果有遗留任务未完成,则通过后台线程来进行恢复。
需要说明的是,图5、图6、图7和图8中所划分的各个阶段可以理解为是完整的操作阶段,不同操作类型所需的变更阶段可以针对性删减,例如:删除操作不需要额外修改数据字典信息。
在一个实施例中,通过查询DDL同步结构可以获取DDL语句执行的情况。通过以下查询语句“select*from information_schema.ddl_job where is_history=1\G”所得到的查询结果可以如下:
mysql>select*from information_schema.ddl_job where is_history=1\G
SCHEMA_NAME:d1
TABLE_NAME:#
TABLE_VERSION:0
DDL_STATUS:0
START_TIMESTAMP:2021-04-07 19:14:14
LAST_TIMESTAMP:2021-04-07 19:14:14
DDL_SQL:create database d1
INFO:{"exec_addr":{"ip":"127.0.0.1","port":9025},"recov_addr":{"ip":"127.0.0.1","port":9025}}
其中,“SCHEMA_NAME”表示目标对象的名称,“TABLE_NAME”表示目标对象中某个表的名称,“TABLE_VERSION”表示表的版本,“DDL_STATUS”表示DDL语句的执行状态,“START_TIMESTAMP”表示执行DDL语句的开始时间,“LAST_TIMESTAMP”表示执行DDL语句的结束时间,“DDL_SQL”为正在执行的DDL语句,“INFO”表示所查询的信息:exec_addr表示DDL执行的机器地址,ip表示对应的IP(Internet Protocol,网际互连协议)地址,port表示对应的网络接口,recov_addr表示DDL执行完成的机器的地址,可以包括正常或异常等状态信息。
本申请还提供一种应用场景,该应用场景应用上述的对象处理方法。具体地,该对象处理方法在该应用场景的应用如下:
本申请的实施例可以应用到分布式数据库系统、计算存储分离的多写数据库集群系统等中。执行一次数据定义语言语句(DDL语句)的流程如图9所示。
首先,执行start_ddl_jod阶段。其次执行push_ddl_job阶段。最后执行end_ddl_job阶段。push_ddl_job 阶段用于实现以下功能:1、推进状态,保证用户数据的可见性;2、记录ddl推进信息,保证在任意节点可恢复。end_ddl_job阶段用于实现信息保存等善后工作。正常结束的情况下,基于是否需要清理临时表而确定是否保留对应的状态记录,另外,用户数据删除需要后台线程执行,也即,需要DDL线程(DDL thread)执行。
如图9所示,当某一阶段执行失败时,可以生成失败记录,并通过恢复线程来恢复执行失败前的状态。其中,执行失败可以是程序执行失败,也可以是断电等系统无法运行的失败情况。进一步地,当无需恢复线程再次处理时,DDL线程回归到原来的状态,并可以处理其他的任务;当需要恢复线程处理时,通过恢复线程来恢复对应的状态,恢复线程可以访问ddl_job来获取失败执行的状态,进而实现状态的恢复。不需要担心DDL失败带来数据问题,一旦DDL执行失败,后台线程会对异常DDL进行恢复,保证数据一致性状态。
对于DDL语句不要求所有计算节点强同步,所有计算节点通过如下方式感知到某一个计算节点上正在进行DDL语句执行:
1、对于DDL语句,通过集群管理节点与存储节点去检验是否有对于同一Schema的变更语句还在执行。因为变更过程是通过任务的形式存储在存储节点的,查询任务的过程实际上就包括检查的过程。另外存储节点的write fence会保证存储节点阻止DDL过程中不允许的读写操作的执行。
2、对于DDL语句中的每一个小阶段完成,都会更新一个集群级别的变量,集群其他节点在开启新的事务时,会基于当前schema变更所处的阶段进行后续执行的逻辑判断,获得集群级别的变量;根据集群级别的变量值来决定是否进入Schema变更判断流程,并判断以下内容:是否当前操作的schema发生了Schema变更,发生了什么变更,目前Schema处于一个什么样的状态,这个状态是否允许当前操作的执行等,处理过程执行完成之后。计算节点的Schema也更新到最新状态。
3、可以在数据字典中增加一个逻辑表,即为前述实施例中的状态记录表。在运行过程中通过同步逻辑表中的状态来确定语句要怎样执行,这个逻辑表可以被计算节点集群中的所有计算节点访问,因此逻辑表中记录的状态可以作为集群级别的变量。其中,逻辑表可以通过ddl_job实现,DDL执行状态可以通过SQL语句的形式进行查询ddl_job,通过ddl_job的记录判定当前DDL是否已经执行完毕,执行到哪一步骤等。另外,逻辑表与数字字典中的同步结构可以不同。
4、假设其他节点在开始执行事务时并没有发生对应的Schema变更,在执行的过程中,可以通过Schema变更过程(Write fence)中对于Schema状态的持久化更新来防止集群中各个节点在执行过程中因为使用旧版本的Schema对象进行操作,而导致的数据不一致问题。增加一个在数据写入存储层时判断Schema对象当前版本是否和操作版本兼容的逻辑来决定是否允许数据读写,进而保证数据读写都不会引入数据的一致性与完整性问题。同时支持多种冲突解决机制。
另外,DDL优先,即DDL总是可以推进执行状态,其他DML语句事务会在某些状态下被回滚。具体地,在其他DML语句执行过程中需要执行DDL语句,则等待DDL语句执行完毕后回滚这些DML语句。
传统技术中,在实现分布式数据库DDL的时候仅仅实现了DDL的串行执行,并且只有特定的节点上才允许执行DDL。本实施例,通过执行阶段的细节划分,能允许在多个节点上执行DDL语句,即,可以在任意计算节点上执行DDL,无关DDL可以并行执行,实现多写。
上述实施例,至少具有以下有益效果:
1、能够保证DDL语句在分布式数据库中执行表现基本与单机MySQL(关系型数据库管理系统)执行DDL语句一致。
2、通过将DDL执行分段,同时允许不同计算节点之间的处理对象版本可以有延迟,最大程度的提升了分布式系统执行DDL过程中,对于DML语句的影响。将传统的DDL语句的执行由原来的从变更前直接变化到变更后的模式转变为多个连续独立的小步骤执行,使得集群中不同计算节点能够通过异步方式感知Schema变更的状态,同时在计算层发起读写操作的时候,存储层会判断当前操作对象的版本状态是否与集群Schema最新状态相符合,防止因为DDL变更导致的读写事务出现数据一致性问题。
3、DDL语句分阶段执行,其他节点根据不同阶段能够及时感知到,并且可以根据DDL的当前执行阶段决定是否事务继续执行/回滚,通过对于状态的精细划分可以将回滚事务减少到最低。通过非锁模式让集群中不同的计算节点感知当前系统正在发生DDL变更语句,相对于强同步实现而言,可以使得DDL语句在执行时对于其他计算节点的影响降到最低。
4、用户数据由write fence进行保障,保证即使有跨版本数据突破了分段DDL的限制,也会被存储层通过write fence发现,并拒绝事务的提交,确保不会读/写错数据。
5、通过ddl_job实现全局控制结构,来跟踪DDL执行信息,后台恢复线程实时监听系统表。保证DDL语句在分布式系统中执行失败时,元数据依然可以恢复到一个正确和一致的状态对用户提供服务。
6、一旦发生DDL失败的场景,后台线程能第一时间根据执行信息进行回滚或推进,保证整个分布式数据库系统能够进入到另外一个一致性状态,并恢复对外提供服务的能力。
应该理解的是,虽然上述流程图中的各个步骤按照箭头的指示依次显示,但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明,这些步骤的执行并没有严格的顺序限制,这些步骤可以以其它的顺序执行。而且,上述流程图中的至少一部分步骤可以包括多个步骤或者多个阶段,这些步骤或者阶段并不必然是在同一时刻执行完成,而是可以在不同的时刻执行,这些步骤或者阶段的执行顺序也不必然是依次进行,而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。
基于与上述实施例中的对象处理方法相同的思想,本申请还提供对象处理装置,该装置可用于执行上述对象处理方法。为了便于说明,对象处理装置实施例的结构示意图中,仅仅示出了与本申请实施例相关的部分,本领域技术人员可以理解,图示结构并不构成对装置的限定,可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件布置。
在一个实施例中,如图10所示,提供了一种对象处理装置1000,该装置可以采用软件模块或硬件模块,或者是二者的结合成为计算机设备的一部分,该装置具体包括:交互请求接收模块1002、变更阶段获取模块1004、冲突校验模块1006和操作执行模块1008,其中:
交互请求接收模块1002,用于接收针对目标对象的对象交互请求。
变更阶段获取模块1004,用于响应于对象交互请求,获取针对目标对象的当前变更操作所在的当前变更阶段;当前变更操作包括多个操作步骤,当前变更操作根据各个操作步骤所对应的冲突操作类型划分为多个变更阶段,冲突操作类型为操作步骤所对应的冲突操作的类型。
冲突校验模块1006,用于根据对象交互请求所对应的目标操作类型以及当前变更阶段对应的当前相关操作类型,确定对象交互请求对应的冲突校验结果目标操作类型,是对象交互请求所请求的目标交互操作的类型。
操作执行模块1008,用于当冲突校验结果为不存在冲突时,根据对象交互请求对目标对象执行目标交互操作。
上述对象处理装置中,通过将变更操作细分为变更阶段,能够在各个变更阶段下进行冲突校验,可以并行运行不存在冲突的操作阶段,同时可以停止运行存在冲突的操作阶段,能在很大程度上降低变更操作对其他交互操作的影响,有效提高对象交互处理的效率。
在一个实施例中,变更阶段获取模块,还包括:操作步骤获取子模块,用于获取针对目标对象的当前变更操作所对应的当前操作步骤;变更阶段确定子模块,用于确定当前操作步骤所对应的变更阶段,作为针对目标对象的当前变更操作所在的当前变更阶段,其中,不同的冲突操作类型的连续操作步骤划分在不同的变更阶段中。
在一个实施例中,对象处理装置,还包括:步骤序列确定模块,用于确定当前变更操作所对应的当前操作步骤序列,当前操作步骤序列包括多个按照操作顺序进行排列的操作步骤;冲突类型确定模块,用于确定当前操作步骤序列中各个操作步骤所对应的冲突操作类型;步骤序列划分模块,用于根据操作步骤所对应的冲突操作类型对当前操作步骤序列进行划分,得到当前变更操作所对应的多个变更阶段,其中,相同冲突操作类型的连续操作步骤划分在相同的变更阶段中。
在一个实施例中,当前相关操作类型包括当前冲突操作类型;冲突校验模块,包括:第一操作类型确定子模块,用于从对象交互请求中提取操作类型标识,根据所提取的操作类型标识得到对象交互请求对应的目标操作类型;第一冲突校验子模块,用于将目标操作类型与当前冲突操作类型进行对比,当根据对比结果确定当前冲突操作类型包括目标操作类型时,确定冲突校验结果为存在冲突。
在一个实施例中,当前相关操作类型包括当前并行操作类型;冲突校验模块,包括:第二操作类型确定子模块,用于从对象交互请求中提取操作类型标识,根据所提取的操作类型标识得到对象交互请求对应的目标操作类型;第二冲突校验子模块,用于将目标操作类型与当前并行操作类型进行对比,当根据对比结果确定当前并行操作类型不包括目标操作类型时,确定冲突校验结果为存在冲突。
在一个实施例中,对象处理装置,还包括:操作停止模块,用于当冲突校验结果为存在冲突时,停止根据对象交互请求对目标对象执行目标交互操作。
在一个实施例中,操作执行模块,包括:步骤序列确定子模块,用于当冲突校验结果为不存在冲突时,响应于对象交互请求,确定目标操作类型所对应的目标操作步骤序列;交互操作执行子模块,用于按照目标操作步骤序列对目标对象执行目标交互操作。
在一个实施例中,交互操作执行子模块,包括:第一操作阶段确定单元,用于确定目标对象对应的第一操作阶段,第一操作阶段为对象交互请求对应的下一操作阶段;锁定请求发送单元,用于当第一操作阶段所对应的阶段优先级满足优先执行条件时,发送针对目标对象的操作锁定请求;操作锁定请求,用于触发正在与目标对象进行交互的计算节点加上对应的对象锁,以锁定在对目标对象的当前操作中,直到解锁;操作步骤执行单元,用于当确定接收到针对操作锁定请求的锁定响应信息时,执行第一操作阶段所对应的操作步骤;锁定响应信息,是计算节点在加上对象锁之后返回的响应信息。
在一个实施例中,对象处理装置,还包括:操作阶段确定模块,用于确定目标对象对应的第二操作阶段,第二操作阶段为第一操作阶段对应的下一操作阶段;对象锁维持模块,用于当第二操作阶段所对应的阶段优先级满足优先执行条件时,维持操作锁定请求所请求的对象锁,执行第二操作阶段所对应的操作步骤。
在一个实施例中,锁定请求发送子模块,包括:内容级别确定单元,用于确定第一操作阶段所对应的操作内容的操作内容级别;锁定请求发送单元,用于当操作内容级别为对象级别时,确定第一操作阶段所对应的阶段优先级满足优先执行条件,发送针对目标对象的操作锁定请求。
在一个实施例中,对象处理装置,还包括:目标版本标识确定模块,用于确定对象交互请求所对应的目标对象版本标识;当前版本标识确定模块,用于获取根据当前变更操作对目标对象进行变更后,目标对象所对应的当前对象版本标识;版本标识对比模块,用于将目标对象版本标识与当前对象版本标识进行对比,当对比不一致时,停止响应对象交互请求。
在一个实施例中,当前相关操作类型包括当前并行操作类型;冲突校验模块,包括:操作类型确定子模块,用于当当前变更阶段为阻塞操作信息写入阶段时,确定当前变更阶段所对应的当前并行操作类型为删除操作类型;阻塞操作信息写入阶段用于写入根据当前变更操作对目标对象进行变更后,目标对象所对应的当前对象版本标识;第三冲突校验子模块,用于当目标操作类型与删除操作类型不匹配时,确定对象交互请求对应的冲突校验结果为存在冲突。
关于对象处理装置的具体限定可以参见上文中对于对象处理方法的限定,在此不再赘述。上述对象处理装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的一个或多个处理器中,也可以以软件形式存储于计算机设备中的存储器中,以便于一个或多个处理器调用执行以上各个模块对应的操作。
在一个实施例中,提供了一种计算机设备,该计算机设备可以是服务器,其内部结构图可以如图11所示。该计算机设备包括通过系统总线连接的一个或多个处理器、存储器和网络接口。其中,该计算机设备的一个或多个处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机可读指令和数据库。该内存储器为非易失性存储介质中的操作系统和计算机可读指令的运行提供环境。该计算机设备的数据库用于存储冲突校验结果等数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机可读指令 被一个或多个处理器执行时以实现一种对象处理方法。
本领域技术人员可以理解,图11中示出的结构,仅仅是与本申请方案相关的部分结构的框图,并不构成对本申请方案所应用于其上的计算机设备的限定,具体的计算机设备可以包括比图中所示更多或更少的部件,或者组合某些部件,或者具有不同的部件布置。
在一个实施例中,还提供了一种计算机设备,包括存储器和一个或多个处理器,存储器中存储有计算机可读指令,该一个或多个处理器执行计算机可读指令时实现上述各方法实施例中的步骤。
在一个实施例中,提供了一个或多个计算机可读存储介质,存储有计算机可读指令,该计算机可读指令被一个或多个处理器执行时实现上述各方法实施例中的步骤。
在一个实施例中,提供了一种计算机程序产品,该计算机程序产品包括计算机可读指令,该计算机可读指令存储在计算机可读存储介质中。计算机设备的一个或多个处理器从计算机可读存储介质读取该计算机可读指令,一个或多个处理器执行该计算机可读指令,使得该计算机设备执行上述各方法实施例中的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程,是可以通过计算机可读指令来指令相关的硬件来完成,所述的计算机可读指令可存储于一非易失性计算机可读取存储介质中,该计算机可读指令在执行时,可包括如上述各方法的实施例的流程。其中,本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用,均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器(Read-Only Memory,ROM)、磁带、软盘、闪存或光存储器等。易失性存储器可包括随机存取存储器(Random Access Memory,RAM)或外部高速缓冲存储器。作为说明而非局限,RAM可以是多种形式,比如静态随机存取存储器(Static Random Access Memory,SRAM)或动态随机存取存储器(Dynamic Random Access Memory,DRAM)等。
以上实施例的各技术特征可以进行任意的组合,为使描述简洁,未对上述实施例中的各个技术特征所有可能的组合都进行描述,然而,只要这些技术特征的组合不存在矛盾,都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式,其描述较为具体和详细,但并不能因此而理解为对发明专利范围的限制。应当指出的是,对于本领域的普通技术人员来说,在不脱离本申请构思的前提下,还可以做出若干变形和改进,这些都属于本申请的保护范围。因此,本申请专利的保护范围应以所附权利要求为准。

Claims (17)

  1. 一种对象处理方法,其特征在于,由计算机设备执行,所述方法包括:
    接收针对目标对象的对象交互请求;
    响应于所述对象交互请求,获取针对所述目标对象的当前变更操作所在的当前变更阶段;所述当前变更操作包括多个操作步骤,所述当前变更操作根据各个所述操作步骤所对应的冲突操作类型划分为多个变更阶段,所述冲突操作类型为所述操作步骤所对应的冲突操作的类型;
    根据所述对象交互请求所对应的目标操作类型以及所述当前变更阶段对应的当前相关操作类型,确定所述对象交互请求对应的冲突校验结果;所述目标操作类型,是所述对象交互请求所请求的目标交互操作的类型;及
    当所述冲突校验结果为不存在冲突时,根据所述对象交互请求对所述目标对象执行所述目标交互操作。
  2. 根据权利要求1所述的方法,其特征在于,所述获取针对所述目标对象的当前变更操作所在的当前变更阶段包括:
    获取针对所述目标对象的所述当前变更操作所对应的当前操作步骤;及
    确定所述当前操作步骤所对应的变更阶段,作为针对所述目标对象的所述当前变更操作所在的所述当前变更阶段,其中,不同的所述冲突操作类型的连续操作步骤划分在不同的所述变更阶段中。
  3. 根据权利要求2所述的方法,其特征在于,对所述当前变更操作进行阶段划分的步骤包括:
    确定所述当前变更操作所对应的当前操作步骤序列,所述当前操作步骤序列包括多个按照操作顺序进行排列的所述操作步骤;
    确定所述当前操作步骤序列中各个所述操作步骤所对应的所述冲突操作类型;及
    根据所述操作步骤所对应的所述冲突操作类型对所述当前操作步骤序列进行划分,得到所述当前变更操作所对应的多个所述变更阶段,其中,相同冲突操作类型的连续操作步骤划分在相同的所述变更阶段中。
  4. 根据权利要求1所述的方法,其特征在于,所述当前相关操作类型包括当前冲突操作类型;所述根据所述对象交互请求所对应的目标操作类型以及所述当前变更阶段对应的当前相关操作类型,确定所述对象交互请求对应的冲突校验结果包括:
    从所述对象交互请求中提取操作类型标识,根据所提取的所述操作类型标识得到所述对象交互请求对应的目标操作类型;及
    将所述目标操作类型与所述当前冲突操作类型进行对比,当根据对比结果确定所述当前冲突操作类型包括所述目标操作类型时,确定所述冲突校验结果为存在冲突。
  5. 根据权利要求1所述的方法,其特征在于,所述当前相关操作类型包括当前并行操作类型;所述根据所述对象交互请求所对应的目标操作类型以及所述当前变更阶段对应的当前相关操作类型,确定所述对象交互请求对应的冲突校验结果包括:
    从所述对象交互请求中提取操作类型标识,根据所提取的所述操作类型标识得到所述对象交互请求对应的目标操作类型;及
    将所述目标操作类型与所述当前并行操作类型进行对比,当根据对比结果确定所述当前并行操作类型不包括所述目标操作类型时,确定所述冲突校验结果为存在冲突。
  6. 根据权利要求1所述的方法,其特征在于,所述方法还包括:
    当所述冲突校验结果为存在冲突时,停止根据所述对象交互请求对所述目标对象执行所述目标交互操作。
  7. 根据权利要求1所述的方法,其特征在于,所述根据所述对象交互请求对所述目标对象执行所述目标交互操作包括:
    确定所述目标对象对应的第一操作阶段,所述第一操作阶段为所述对象交互请求对应的下一操作阶段;
    当所述第一操作阶段所对应的阶段优先级满足优先执行条件时,发送针对所述目标对象的操作锁 定请求;所述操作锁定请求,用于触发正在与所述目标对象进行交互的计算节点加上对应的对象锁,以锁定在对所述目标对象的当前操作中,直到解锁;及
    当确定接收到针对所述操作锁定请求的锁定响应信息时,执行所述第一操作阶段所对应的所述操作步骤;所述锁定响应信息,是所述计算节点在加上所述对象锁之后返回的响应信息。
  8. 根据权利要求7所述的方法,其特征在于,所述方法还包括:
    确定所述目标对象对应的第二操作阶段,所述第二操作阶段为所述第一操作阶段对应的下一操作阶段;及
    当所述第二操作阶段所对应的阶段优先级满足所述优先执行条件时,维持所述操作锁定请求所请求的对象锁,执行所述第二操作阶段所对应的所述操作步骤。
  9. 根据权利要求7所述的方法,其特征在于,所述当所述第一操作阶段所对应的阶段优先级满足优先执行条件时,发送针对所述目标对象的操作锁定请求包括:
    确定所述第一操作阶段所对应的操作内容的操作内容级别;及
    当所述操作内容级别为对象级别时,确定所述第一操作阶段所对应的阶段优先级满足优先执行条件,发送针对所述目标对象的操作锁定请求。
  10. 根据权利要求1所述的方法,其特征在于,所述方法还包括:
    确定所述对象交互请求所对应的目标对象版本标识;
    获取根据所述当前变更操作对所述目标对象进行变更后,所述目标对象所对应的当前对象版本标识;及
    将所述目标对象版本标识与所述当前对象版本标识进行对比,当对比不一致时,停止响应所述对象交互请求。
  11. 根据权利要求1所述的方法,其特征在于,所述当前相关操作类型包括当前并行操作类型;所述根据所述对象交互请求所对应的目标操作类型以及所述当前变更阶段对应的当前相关操作类型,确定所述对象交互请求对应的冲突校验结果包括:
    当所述当前变更阶段为阻塞操作信息写入阶段时,确定所述当前变更阶段所对应的当前并行操作类型为删除操作类型;所述阻塞操作信息写入阶段用于写入根据所述当前变更操作对所述目标对象进行变更后,所述目标对象所对应的当前对象版本标识;及
    当所述目标操作类型与所述删除操作类型不匹配时,确定所述对象交互请求对应的冲突校验结果为存在冲突。
  12. 根据权利要求1至11中任一项所述的方法,其特征在于,所述当所述冲突校验结果为不存在冲突时,根据所述对象交互请求对所述目标对象执行所述目标交互操作,包括:
    当所述冲突校验结果为不存在冲突时,响应于所述对象交互请求,确定所述目标操作类型所对应的目标操作步骤序列;
    按照所述目标操作步骤序列对所述目标对象执行所述目标交互操作。
  13. 一种对象处理装置,其特征在于,由计算机设备执行,所述装置包括:
    交互请求接收模块,用于接收针对目标对象的对象交互请求;
    变更阶段获取模块,用于响应于所述对象交互请求,获取针对所述目标对象的当前变更操作所在的当前变更阶段;所述当前变更操作包括多个操作步骤,所述当前变更操作根据各个所述操作步骤所对应的冲突操作类型划分为多个变更阶段,所述冲突操作类型为所述操作步骤所对应的冲突操作的类型;
    冲突校验模块,用于根据所述对象交互请求所对应的目标操作类型以及所述当前变更阶段对应的当前相关操作类型,确定所述对象交互请求对应的冲突校验结果;所述目标操作类型,是所述对象交互请求所请求的目标交互操作的类型;及
    操作执行模块,用于当所述冲突校验结果为不存在冲突时,根据所述对象交互请求对所述目标对象执行所述目标交互操作。
  14. 根据权利要求13所述的装置,其特征在于,变更阶段获取模块,还包括:
    操作步骤获取子模块,用于获取针对所述目标对象的所述当前变更操作所对应的当前操作步骤;及
    变更阶段确定子模块,用于确定所述当前操作步骤所对应的变更阶段,作为针对所述目标对象的所述当前变更操作所在的所述当前变更阶段,其中,不同的所述冲突操作类型的连续操作步骤划分在不同的所述变更阶段中。
  15. 一种计算机设备,包括存储器和一个或多个处理器,所述存储器存储有计算机可读指令,其特征在于,所述一个或多个处理器执行所述计算机可读指令时实现权利要求1至12中任一项所述方法的步骤。
  16. 一个或多个计算机可读存储介质,存储有计算机可读指令,其特征在于,所述计算机可读指令被一个或多个处理器执行时实现权利要求1至12中任一项所述方法的步骤。
  17. 一种计算机程序产品,包括计算机可读指令,其特征在于,所述计算可读指令被一个或多个处理器执行时实现权利要求1至12中任一项所述的方法的步骤。
PCT/CN2022/086091 2021-05-19 2022-04-11 对象处理方法、装置、计算机设备和存储介质 WO2022242372A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP22803697.6A EP4239492A4 (en) 2021-05-19 2022-04-11 OBJECT PROCESSING METHOD AND APPARATUS, COMPUTER DEVICE AND STORAGE MEDIUM
JP2023523281A JP7566146B2 (ja) 2021-05-19 2022-04-11 オブジェクト処理方法、装置、及びコンピュータ機器
US18/062,308 US20230098963A1 (en) 2021-05-19 2022-12-06 Object processing method and apparatus, computer device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110547589.7 2021-05-19
CN202110547589.7A CN112988777B (zh) 2021-05-19 2021-05-19 对象处理方法、装置、计算机设备和存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/062,308 Continuation US20230098963A1 (en) 2021-05-19 2022-12-06 Object processing method and apparatus, computer device, and storage medium

Publications (1)

Publication Number Publication Date
WO2022242372A1 true WO2022242372A1 (zh) 2022-11-24

Family

ID=76336824

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/086091 WO2022242372A1 (zh) 2021-05-19 2022-04-11 对象处理方法、装置、计算机设备和存储介质

Country Status (5)

Country Link
US (1) US20230098963A1 (zh)
EP (1) EP4239492A4 (zh)
JP (1) JP7566146B2 (zh)
CN (1) CN112988777B (zh)
WO (1) WO2022242372A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112988777B (zh) * 2021-05-19 2021-08-10 腾讯科技(深圳)有限公司 对象处理方法、装置、计算机设备和存储介质
CN117201229B (zh) * 2023-11-07 2024-02-20 横川机器人(深圳)有限公司 一种通讯方法、系统及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109857745A (zh) * 2019-02-20 2019-06-07 平安科技(深圳)有限公司 数据库操作控制方法、装置、计算机设备和存储介质
CN110909012A (zh) * 2019-12-04 2020-03-24 上海达梦数据库有限公司 数据库对象的封锁方法、装置、设备和存储介质
CN111338766A (zh) * 2020-03-12 2020-06-26 腾讯科技(深圳)有限公司 事务处理方法、装置、计算机设备及存储介质
CN112988777A (zh) * 2021-05-19 2021-06-18 腾讯科技(深圳)有限公司 对象处理方法、装置、计算机设备和存储介质

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3937371B2 (ja) * 1998-05-08 2007-06-27 富士通株式会社 競合制御方法及び競合制御システム
JP4158534B2 (ja) * 2003-01-21 2008-10-01 修平 西山 分散型データベースシステム
JP2008165370A (ja) 2006-12-27 2008-07-17 Internatl Business Mach Corp <Ibm> オンライントランザクション処理を分割し、分散環境で実行するための方法および装置。
CN101184106A (zh) 2007-11-30 2008-05-21 北京航空航天大学 一种移动数据库的关联事务处理方法
US20100082652A1 (en) * 2008-09-29 2010-04-01 Chacha Search, Inc. Method and system for managing user interaction
CN102217357B (zh) * 2011-05-20 2013-10-09 华为技术有限公司 用于控制业务的方法及其装置和系统
US20140129531A1 (en) 2012-11-07 2014-05-08 Nec Laboratories America, Inc. Automatic Entity-Grouping for Scalable OLTP
CN104678769B (zh) * 2014-12-31 2019-12-31 海尔优家智能科技(北京)有限公司 一种Event-Action冲突解决方法及装置
CN111949315A (zh) 2019-05-16 2020-11-17 富士通株式会社 用于区块链账本数据的管理装置和方法
CN110795709B (zh) * 2019-10-31 2022-08-12 北京达佳互联信息技术有限公司 一种进行业务操作的方法、装置和电子设备及存储介质
CN112216365A (zh) * 2020-10-12 2021-01-12 微医云(杭州)控股有限公司 对目标事件的评估方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109857745A (zh) * 2019-02-20 2019-06-07 平安科技(深圳)有限公司 数据库操作控制方法、装置、计算机设备和存储介质
CN110909012A (zh) * 2019-12-04 2020-03-24 上海达梦数据库有限公司 数据库对象的封锁方法、装置、设备和存储介质
CN111338766A (zh) * 2020-03-12 2020-06-26 腾讯科技(深圳)有限公司 事务处理方法、装置、计算机设备及存储介质
CN112988777A (zh) * 2021-05-19 2021-06-18 腾讯科技(深圳)有限公司 对象处理方法、装置、计算机设备和存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4239492A4 *

Also Published As

Publication number Publication date
JP7566146B2 (ja) 2024-10-11
EP4239492A1 (en) 2023-09-06
CN112988777A (zh) 2021-06-18
US20230098963A1 (en) 2023-03-30
EP4239492A4 (en) 2024-07-31
CN112988777B (zh) 2021-08-10
JP2023546897A (ja) 2023-11-08

Similar Documents

Publication Publication Date Title
CN111338766B (zh) 事务处理方法、装置、计算机设备及存储介质
US11429641B2 (en) Copying data changes to a target database
US11176171B2 (en) Controlling a multi-database system
US10503699B2 (en) Metadata synchronization in a distrubuted database
JP7546832B2 (ja) トランザクション処理方法、装置、コンピュータ機器及びコンピュータプログラム
CN111597015B (zh) 事务处理方法、装置、计算机设备及存储介质
WO2022242372A1 (zh) 对象处理方法、装置、计算机设备和存储介质
JP7438603B2 (ja) トランザクション処理方法、装置、コンピュータデバイス及びコンピュータプログラム
US11599514B1 (en) Transactional version sets
US20060184528A1 (en) Distributed database with device-served leases
WO2020119050A1 (en) Write-write conflict detection for multi-master shared storage database
US20210326359A1 (en) Compare processing using replication log-injected compare records in a replication environment
CN109902127B (zh) 历史态数据处理方法、装置、计算机设备及存储介质
US11003550B2 (en) Methods and systems of operating a database management system DBMS in a strong consistency mode
US20230305724A1 (en) Data management method and apparatus, computer device, and storage medium
WO2023065868A1 (zh) 事务执行方法、装置、计算设备及存储介质
US11886422B1 (en) Transactional protocol for snapshot isolation without synchronized clocks
Padhye et al. Scalable transaction management with snapshot isolation for NoSQL data storage systems
US20240028598A1 (en) Transaction Processing Method, Distributed Database System, Cluster, and Medium
US20230110826A1 (en) Log execution method and apparatus, computer device and storage medium
CN117539841B (zh) 一种分布式文件系统元数据管理系统及其操作方法
US11709809B1 (en) Tree-based approach for transactionally consistent version sets
US11940972B2 (en) Execution of operations on partitioned tables
CN116089359A (zh) 数据库快照的生成方法、装置、电子设备和介质
CN118369660A (zh) 用于控制数据库集群中的节点的数据管理设备和方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22803697

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023523281

Country of ref document: JP

ENP Entry into the national phase

Ref document number: 2022803697

Country of ref document: EP

Effective date: 20230531

NENP Non-entry into the national phase

Ref country code: DE