CN115905274A - Data processing method and device, electronic equipment and medium - Google Patents
Data processing method and device, electronic equipment and medium Download PDFInfo
- Publication number
- CN115905274A CN115905274A CN202110919758.5A CN202110919758A CN115905274A CN 115905274 A CN115905274 A CN 115905274A CN 202110919758 A CN202110919758 A CN 202110919758A CN 115905274 A CN115905274 A CN 115905274A
- Authority
- CN
- China
- Prior art keywords
- data processing
- entity object
- data
- processing operation
- request data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The present disclosure relates to a data processing method, apparatus, electronic device, and medium, which can be applied to the technical field of internet and cloud service, and the method includes: receiving request data to be processed and a corresponding data processing operation instruction; analyzing the request data into a logic entity object in a metadata field; in the metadata field, performing data processing operation on the logical entity object according to a data processing operation instruction corresponding to the request data; and converting the data processing operation aiming at the logic entity object into an operation aiming at a physical object in a database to be implemented to obtain the processing result of the request data.
Description
Technical Field
The present disclosure relates to the field of internet technologies and cloud service technologies, and in particular, to a data processing method and apparatus, an electronic device, and a medium.
Background
With the development of internet technology, each organization can generate massive business data, internal operation and maintenance data and the like in the business operation process. Different organizations have implemented storage and operational management of large amounts of data by introducing cloud services. When application development is performed based on a user browser side/server side (B/S) architecture, mostly Java objects (POJO) are transmitted through a Model View Controller (MVC) software architecture so as to perform data interaction and circulation, and a backend performs database operation through a corresponding database interface. At present, the model of the database needs to be adapted to various objects, so that corresponding database model design needs to be performed for different objects.
In the course of implementing the disclosed concept, the inventors found that there are at least the following technical problems in the related art: in the related art, in the process of converting a metadata object into an SQL statement to perform database operation, an annotation mode is mostly adopted to perform annotation of Bean (a reusable class in java), and then based on an SQL conversion tool, annotation information is analyzed through reflection of the class to complete conversion of the SQL statement; in this way, because the annotation is a cold coding mode, the business logic and the conversion logic are fixed, and the annotation information of the dynamically expanded Bean cannot be configured according to the database; in addition, by adopting the annotation mode to convert the metadata, if the business expansion requirement exists in the subsequent process, research and development personnel need to customize and develop again, so that the flexibility and the expandability are lost.
Disclosure of Invention
In order to solve the technical problem or at least partially solve the technical problem, embodiments of the present disclosure provide a data processing method, apparatus, electronic device, and medium.
In a first aspect, an embodiment of the present disclosure provides a data processing method. The method comprises the following steps: receiving request data to be processed and a corresponding data processing operation instruction; analyzing the request data into a logic entity object in the metadata field; in the metadata field, according to the data processing operation instruction corresponding to the request data, performing data processing operation on the logic entity object; and converting the data processing operation aiming at the logic entity object into an operation aiming at a physical object in a database to be implemented to obtain the processing result of the request data.
According to an embodiment of the present disclosure, the parsing the request data into the logical entity object in the metadata field includes: and resolving the request data into a logic entity object of a metadata field based on an interceptor or a filter before the request data is processed.
According to an embodiment of the present disclosure, in the metadata field, the performing, with respect to the logical entity object, a data processing operation corresponding to the request data includes: correspondingly distributing the logic entity object obtained by analyzing the request data to the execution instruction of the request data; and in the process that the request data is executed according to the execution instruction, carrying out data processing operation aiming at the logic entity object in the metadata field according to the data processing operation instruction corresponding to the request data.
According to the embodiment of the disclosure, the metadata field stores a preconfigured mapping relationship, and the mapping relationship is used for representing association between a logical entity object in the logical table and a physical object in the physical table. The converting the data processing operation for the logical entity object into an operation for a physical object in the database includes: converting the data operation aiming at the logic entity object into a target operation aiming at a physical object in a database based on the mapping relation; and implementing corresponding target operation aiming at the physical object in the database.
According to an embodiment of the present disclosure, the converting the data operation for the logical entity object into the target operation for the physical object in the database based on the mapping relationship includes: determining a target position in a target physical table associated with the logical table position where the logical entity object is located based on the mapping relation; and converting the data operation aiming at the logic entity object into an operation to be implemented aiming at the physical object at the target position of the target physical table.
According to the embodiment of the disclosure, the request data is associated with a user identifier, and data processing operations of different users are isolated based on the user identifier. Parsing the request data into a logical entity object in the metadata field, including: in the metadata field, the request data is parsed into: a logical entity object associated with the user identifier; according to the data processing operation instruction corresponding to the request data, the method for implementing data processing operation for the logic entity object comprises the following steps: according to a data processing operation instruction corresponding to the request data, carrying out data processing operation on the logic entity object associated with the user identifier; converting the data processing operation aiming at the logic entity object into an operation aiming at a physical object in a database for implementation, wherein the operation comprises the following steps: and converting the data processing operation aiming at the logic entity object associated with the user identification into the operation aiming at the physical object associated with the user identification in the database for implementation.
According to an embodiment of the present disclosure, the method further includes: precompiled candidate SQL statements for database operations; attribute values of the operation objects and/or the operation conditions of the candidate SQL statements are transmitted when performing data processing operations on the request data, or when performing operations on the physical object.
According to an embodiment of the present disclosure, the performing, according to the data processing operation instruction corresponding to the request data, a data processing operation on the logical entity object includes: determining a target candidate SQL statement matched with the type based on the type of the data processing operation instruction corresponding to the request data; transmitting the attribute values of the logic entity object and the corresponding data processing operation instruction into the target candidate SQL statement to generate a logic operation SQL statement; and based on the logic operation SQL statement, carrying out data processing operation on the logic entity object. The converting the data processing operation for the logical entity object into an operation for a physical object in a database, includes: converting the logic entity object of the logic operation SQL statement into a physical object to generate a physical operation SQL statement; and implementing corresponding operation for the physical object in the database based on the physical operation SQL statement.
In a second aspect, embodiments of the present disclosure provide an apparatus for data processing. The above-mentioned device includes: the device comprises a receiving module, an analysis module, a logic operation module and a physical operation module. The receiving module is used for receiving the request data to be processed and the corresponding data processing operation instruction. The analysis module is used for analyzing the request data into a logic entity object in the metadata field. The logic operation module is configured to perform a data processing operation on the logic entity object in the metadata field according to a data processing operation instruction corresponding to the request data. The physical operation module is used for converting the data processing operation aiming at the logic entity object into the operation aiming at the physical object in the database to implement, and obtaining the processing result of the request data.
In a third aspect, embodiments of the present disclosure provide an electronic device. The electronic equipment comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus; a memory for storing a computer program; a processor for implementing the data processing method described above when executing the program stored in the memory.
In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium. The computer-readable storage medium has stored thereon a computer program which, when executed by a processor, implements the method of data processing as described above.
Compared with the prior art, the technical scheme provided by the embodiment of the disclosure at least has part or all of the following advantages:
the method is characterized in that request data and corresponding data processing operation instructions are analyzed into logical entity objects and corresponding data processing operations in a metadata field, the method is applicable to service data access and transmission under various scenes at the front end, data processing operations are carried out on the logical entity objects at the back end in a unified mode, and meanwhile, a conversion process of a logical table and a physical table can be carried out in the metadata field: the data processing operation aiming at the logic entity object is converted into the operation aiming at the physical object in the database to be implemented, and then the database persistence is realized, so that the management operation based on the metadata can realize the generalized data processing process of the business data of various types/scenes, is suitable for the construction of various business systems (applications), can quickly respond to various types of business expansion requirements, and has wide applicability.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the related art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1A schematically illustrates a system architecture of a method and apparatus for data processing suitable for use with embodiments of the present disclosure;
FIG. 1B schematically illustrates the architecture of a metadata model suitable for use in the methods of database operation of embodiments of the present disclosure;
FIG. 2 schematically shows a flow diagram of a method of data processing according to an embodiment of the present disclosure;
FIG. 3 schematically illustrates a functional architecture diagram of a metadata domain according to an embodiment of the present disclosure;
FIG. 4 schematically illustrates a detailed implementation process diagram of a method of data processing according to an embodiment of the present disclosure;
FIG. 5 schematically shows a flow diagram of a method of data processing according to another embodiment of the present disclosure;
fig. 6 schematically shows a detailed implementation flowchart of operation S203 according to an embodiment of the present disclosure;
fig. 7 schematically shows a detailed implementation flowchart of operation S204 according to an embodiment of the present disclosure;
fig. 8 schematically shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure; and
fig. 9 schematically shows a block diagram of an electronic device provided in an embodiment of the present disclosure.
Detailed Description
Take the architecture of a cloud service (ABS cloud service for short) corresponding to an ABS asset securitization system (an application on a terminal device) as an example. In the related art, when a database model corresponding to the ABS asset securitization system is accessed, the following operations need to be repeated every time a new asset is accessed.
Firstly, researching a new asset type, carrying out asset dimension database modeling according to the new asset type, and designing a database asset table structure.
Then, the developer writes Java classes of the relevant Java objects (POJO, plain orindary Java Object) according to the database model.
Then, the research and development personnel follow up the business process and receive the asset data in the Json format in the service interface, and simultaneously convert the asset data into a Java Bean (a reusable class in Java), and then perform basic operation of the Bean according to the related business process.
And finally, persisting the operated Bean object data into a new asset table of the database.
Meanwhile, operations such as increasing, deleting, modifying, checking and the like of the database also need to be set in a targeted manner. In the related art, in the process of converting a metadata object into an SQL statement to perform database operation, an annotation mode is mostly adopted to perform annotation of Bean (a reusable class in java), and then based on an SQL conversion tool, annotation information is analyzed through reflection of the class to complete conversion of the SQL statement; in this way, because the annotation is a cold coding mode, the business logic and the conversion logic are fixed, and the annotation information of the dynamically expanded Bean cannot be configured according to the database; in addition, by adopting the annotation mode to perform metadata conversion, if a business expansion requirement exists in the subsequent process, research and development personnel need to customize and develop again, so that the flexibility and the expandability are lost.
In view of this, embodiments of the present disclosure provide a data processing method, apparatus, electronic device and medium. The method comprises the following steps: receiving request data to be processed and a corresponding data processing operation instruction; analyzing the request data into a logic entity object in a metadata field; in the metadata field, performing data processing operation on the logical entity object according to a data processing operation instruction corresponding to the request data; and converting the data processing operation aiming at the logic entity object into an operation aiming at a physical object in a database to be implemented to obtain the processing result of the request data.
To make the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions in the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some, but not all, embodiments of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
Fig. 1A schematically illustrates a system architecture of a method and apparatus for data processing suitable for use in embodiments of the present disclosure.
Referring to fig. 1A, a system architecture 100 suitable for the method and apparatus for data processing according to the embodiment of the present disclosure includes: terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use terminal devices 101, 102, 103 to interact with a server 105 over a network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have installed thereon various communication client applications (application programs are simply referred to as applications), such as ABS asset securitization system applications, and may also include other client applications, such as shopping-type applications, web browser applications, search-type applications, instant messaging tools, mailbox clients, social platform software, and the like (for example only).
The terminal devices 101, 102, 103 may be various electronic devices that display screens and support web browsing, such as electronic devices including but not limited to smartphones, tablets, laptops, desktop computers, smartwatches, and the like.
The server 105 may be a server providing various services, such as a background management server (for example only) providing service support for data processing of an application interface or a website browsed by a user using the terminal device 101, 102, 103. The background management server may perform data analysis and processing on the received data processing request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to a user request) to the terminal device. The server can access the database through a network interface or carry the database service by itself.
It should be noted that the data processing method provided by the embodiment of the present disclosure may be generally executed by the server 105 or a terminal device with certain computing capability, where the terminal device may be a networked terminal device providing a cloud service. Accordingly, the data processing apparatus provided by the embodiment of the present disclosure may be generally disposed in the server 105 or the terminal device with a certain computing capability. The method for processing data provided by the embodiments of the present disclosure may also be performed by a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the data processing apparatus provided by the embodiment of the present disclosure may also be disposed in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.
It should be understood that the number of terminal devices, networks, and servers in FIG. 1A are merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for an implementation.
FIG. 1B schematically illustrates an architecture of a metadata model suitable for use in methods of data processing according to embodiments of the present disclosure.
Referring to fig. 1B, a metadata model 110 suitable for use in the method of data processing of the embodiments of the present disclosure is mainly divided into three layers: the first layer is a physical model layer 111 which is a real physical table structure and model of the bottom layer of the database; the second layer is a mapping relation layer 112, which represents the corresponding relation between the physical table and the logical table, and is a one-to-one mapping relation (1:1); the third layer is a Logic model layer 113a and a Logic metadata (Logic _ Meta) layer 113b, which mainly describe the POJO object of the business model design, and can be used for various business operations. The dashed arrows indicate that the mapping relationship between layers is one-to-one (1:1) mapping or one-to-many (1:N) mapping. The user can be, for example, a tenant of the ABS cloud service, the types of data objects of the tenant are N, N ≧ 2, and there are N corresponding physical model layers.
Referring to FIG. 1B, the metadata model 110 may further include a logical model relationship (Logic _ relationship) layer 114 for characterizing relationships and references between the logical model and the logical model.
Referring to fig. 1B, the metadata model 110 may further include a logic layer 115 corresponding to an operation instruction of a user, and is used to implement operations such as adding, deleting, modifying, and checking the database.
An application scenario of the method for data processing provided by the embodiment of the present disclosure is exemplarily described below with reference to fig. 1A and 1B.
As shown in fig. 1A and fig. 1B, in the process that a user uses an application on a terminal device 101, 102, or 103, a data operation request is generated, and a front end (for example, a browser end of the terminal device) sends request data to be processed and a corresponding data processing operation instruction to a server or a terminal device providing cloud service. Accordingly, the request data to be processed and the corresponding data processing operation instruction are received at the server or the terminal device providing the cloud service (corresponding to the subsequent operation S201). Then, the above-mentioned request data to be processed is parsed into a logical entity object of the metadata domain (corresponding to the subsequent operation S202), and by uniformly converting the request data into an object of the metadata domain, the object can be used to perform a logical operation of the front end, which is described as a logical entity object. During the data processing, a data processing operation (for example, an operation such as an add/delete/modify/check) is performed on the logical entity object in the metadata domain according to the data processing operation command corresponding to the request data (corresponding to the subsequent operation S203). At the application level, the server 105 is accessed through the network 104, and the business process is supported and implemented based on the database service provided by the database, and the business data is not submitted to the database for persistence at this time. During the data processing (or described as implementing a business method/process), corresponding data processing operations are implemented on the logical entity objects of the logical table based on the logical layer 115, the logical model layer 113a, and the logical metadata layer 113 b. In the process of database persistence, the data processing operation on the logical entity object is converted into an operation on a physical object in the database to be implemented (corresponding to operation S204), and a processing result of the request data is obtained. Therefore, the method and the device can be suitable for service data access and transmission under various scenes at the front end, perform data processing operation on the logic entity object at the back end in a unified mode, and simultaneously realize the conversion process of the logic table and the physical table in the metadata field.
Embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.
A first exemplary embodiment of the present disclosure provides a method of data processing.
Fig. 2 schematically shows a flow diagram of a method of data processing according to an embodiment of the present disclosure.
Referring to fig. 2, a method for data processing provided by an embodiment of the present disclosure includes the following operations: s201, S202, S203 and S204. Operations S201 to S204 may be performed by a server or by a terminal device capable of providing a cloud service.
In operation S201, request data to be processed and a corresponding data processing operation instruction are received.
In operation S202, the request data is parsed into logical entity objects in the metadata domain.
In operation S203, in the metadata field, a data processing operation is performed on the logical entity object according to a data processing operation command corresponding to the request data.
In operation S204, the data processing operation on the logical entity object is converted into an operation on a physical object in the database, and the processing result of the request data is obtained.
In operation S201, the front end (for example, a browser end of the terminal device) sends the to-be-processed request data and the corresponding data processing operation instruction to the server or the terminal device providing the cloud service. Correspondingly, the request data to be processed and the corresponding data processing operation instruction are received on a server or terminal equipment providing cloud service. The request data can be in a Json message format corresponding to http, and currently, the front-end interaction and the back-end interaction commonly adopt the Json message format and can be adapted to most scenes. It should be understood that the request data may also be in a custom format, such as an XML format or other custom format.
The operation information carried by the data processing operation instruction comprises operations of increasing, deleting, modifying, checking and the like executed in the database.
In operation S202, the metadata field (Logic-Region) is equivalent to a container for holding data, and the data in the container has general processing Logic, wherein a logical entity object in the metadata field can be used to perform a logical operation on a front end, and can be converted into physical SQL by a standardized conversion tool to implement a persistence operation on the database.
Fig. 3 schematically illustrates a functional architecture diagram of a metadata domain according to an embodiment of the present disclosure.
Referring to fig. 3, the metadata field 300 includes at least some of the following objects: a logical Entity object (Entity) 301, a user (e.g., a Tenant of a cloud service, tenant) 302, a domain operation type (SqlCommandType) 303, a Page (Page) 304, a column set (Fields) 305, an operation logical table name (Tablename) 306, condition information (Condition) 307, and a precompiled object (PreparedParam) 308.
The logical Entity object (Entity) 301 corresponds to an object when a data processing operation is performed in the metadata field. An entity for storing an object in the metadata field is a JSON object, stores operation process data of a front end, a back end and a subsequent business process, has a structure of a Key-Value pair (Key-Value) model, and represents an object body in the operation process of the subsequent metadata field, wherein the object body is used for operation objects of the business processes such as Insert/Update and the like.
The user 302 is, for example, a Tenant of the cloud service, and the user identifier is associated with the request data and the data processing operation executed correspondingly during the data interaction process, so as to implement physical and logical isolation between the late data and the data processing service. Meanwhile, the information of the tenant is synchronized to the SQL rewrite (the SQL statement is rewritten into a physical operation SQL statement by a logic operation SQL statement) during the database persistence operation.
The domain operation type (SqlCommandType) 303 is mainly used as an operation tag of the domain, and includes operations such as query (Select)/change (Update)/add/Insert (Insert)/Delete), and processing operations are performed on a specific object by the tag of the domain operation type.
The paging (Page) 304 object represents the paging condition of the query operation in the object interaction process in the metadata field, and comprises an Order object (Order) and an element (Size/Number) for paging conversion of the front-end query service.
Column collections (Fields) 305, an object used as an object definition for returned data in the context of a query, contains mainly a collection of keys of JSON entities that return objects, used as keys for query operations in Select types.
The operation logical table name (Tablename) 306, identifies the logical object in the metadata domain, the key label for subsequent logical table and physical table translation, and the information corresponding to the logical table of the physical operation SQL statement.
Condition information (Condition) 307, corresponding to where Condition in the regular logical operation SQL statement, contains Condition information for filtering/filtering, facilitating data filtering preparation for Select operation in later conversion into physical operation SQL.
The precompiled object (precompiled param) 308 is used for precompiling an ordered object (corresponding to the subsequent sub operations S2031 to S2033 and S2041 to S2042) of a storage element value in the SQL conversion (converted from a logical operation SQL statement to a physical operation SQL statement) directly according to an attribute value of an Entity or Condition object in the metadata field in the process of converting the physical SQL statement at the later stage, and is helpful for improving the efficiency of data processing based on the setting of the precompiled object.
In the above operation S202, various types of request data transmitted from the front end can be converted into a unified object in the metadata domain, and processed by using a common processing logic: in the process of data processing, a data processing operation corresponding to the request data is performed on the logical entity object in the metadata domain (corresponding to operation S203); in the process of database persistence, the data processing operation for the above logical entity object is converted into an operation for a physical object in the database to be performed (corresponding to operation S204).
Based on the above operations S201 to S204, by parsing the request data and the corresponding data processing operation command into the logical entity object and the corresponding data processing operation in the metadata domain, the front end is applicable to service data access and transmission in various scenes, and the back end performs the data processing operation on the logical entity object in a unified manner, and at the same time, by performing a conversion process between the logical table and the physical table in the metadata domain: the data processing operation aiming at the logic entity object is converted into the operation aiming at the physical object in the database to be implemented, and then the database persistence is realized, so that the management operation based on the metadata can realize the generalized data processing process of the business data of various types/scenes, is suitable for the construction of various business systems (applications), can quickly respond to various types of business expansion requirements, and has wide applicability.
Fig. 4 schematically shows a detailed implementation process diagram of a method of data processing according to an embodiment of the present disclosure.
Referring to fig. 4, in operation S201, a front-end request is received to simplify a process of schematically receiving request data to be processed and a corresponding data processing operation instruction.
The front-end http request data may be in a json type message format. Examples of the content of the request data and the corresponding data processing operation instructions are: "user code user _ id, user name user _ name, user age, user address user _ addr," operation of executing query (select), the query condition is: user _ id >12". The query condition may be in other forms, such as "user _ name = lisa" or "age =18 to 30", and it should be understood that a plurality of query conditions may be combined, and a plurality of query conditions may have a logical relationship, such as "or", "and".
For example, the request data and corresponding data processing operation instructions are in the form of:
or in the form of:
or in the form of:
referring to fig. 4, according to an embodiment of the present disclosure, in the operation S202, parsing the request data into a logical entity object in a metadata field includes: before the request data is processed, the request data is resolved into a logical entity object of a metadata field based on an interceptor or a filter.
Correspondingly, the logic entity object for analyzing the request data into the metadata field is in the following form:
or in the form of:
or in the form of:
referring to fig. 4, according to an embodiment of the present disclosure, in the operation S203, in the metadata field, the performing, on the logical entity object, a data processing operation corresponding to the request data includes the following: s2031 and S2032.
In sub-operation S2031, the logical entity object obtained by parsing the request data is correspondingly distributed to the execution instruction of the request data.
In sub-operation S2032, during the process of executing the request data according to the execution instruction, a data processing operation is performed on the logical entity object in the metadata field according to a data processing operation instruction corresponding to the request data.
The execution instruction includes a data processing operation instruction.
According to the embodiment of the disclosure, the metadata field stores a preconfigured mapping relationship, and the mapping relationship is used for representing association between a logical entity object in the logical table and a physical object in the physical table.
Referring to fig. 4, in the operation S204, converting the data processing operation for the logical entity object into an operation for a physical object in a database to be implemented includes the following sub-operations: s2041 and S2042.
In sub-operation S2041, based on the mapping relationship, the data operation for the logical entity object is converted into a target operation for a physical object in the database.
In sub-operation S2042, a corresponding target operation is performed on the physical object in the database.
According to an embodiment of the present disclosure, in the sub-operation S2041, based on the mapping relationship, converting the data operation for the logical entity object into a target operation for a physical object in a database includes: determining a target position in a target physical table associated with the logical table position where the logical entity object is located based on the mapping relation; and converting the data operation aiming at the logic entity object into an operation to be implemented aiming at the physical object at the target position of the target physical table.
The mapping relation is configured in advance in the data storage process. And according to the data type information of the elements in the logical table, corresponding the elements at the specific positions in the logical table with the elements at the specific positions in the physical table.
According to the embodiment of the disclosure, the request data is associated with a user identifier, and data processing operations of different users are isolated based on the user identifier. For different types of request data, based on the association of the user identifiers, the different types of request data can be processed under the same processing logic, and each user can perform data processing on the corresponding data without interfering with the data of other users. Therefore, when the cloud service meets the requirements of various types of users, processing can be carried out on the basis of a uniform framework, and data operation isolation among the users is realized.
Then, in operation S202, parsing the request data into a logical entity object in the metadata domain includes: in the metadata field, the request data is parsed into: and the logic entity object is associated with the user identification.
In operation S203, the performing, on the logical entity object, a data processing operation corresponding to the request data includes: and implementing data processing operation corresponding to the request data aiming at the logic entity object associated with the user identifier.
In operation S204, converting the data processing operation on the logical entity object into an operation on a physical object in a database, and implementing the operation includes: and converting the data processing operation aiming at the logical entity object associated with the user identifier into the operation aiming at the physical object associated with the user identifier in the database for implementation.
Fig. 5 schematically shows a flow chart of a method of data processing according to another embodiment of the present disclosure.
Referring to fig. 5, the method for processing data according to the embodiment of the present disclosure includes, in addition to the above operations S201 to S204, the following operations S501: candidate SQL statements for database operations are precompiled. Attribute values of the operands and/or operating conditions of the candidate SQL statement are either passed in when performing data processing operations on the requested data or passed in when performing operations on the physical object.
Fig. 6 schematically shows a detailed implementation flowchart of operation S203 according to an embodiment of the present disclosure.
According to an embodiment of the present disclosure, referring to fig. 6, in the operation S203, according to the data processing operation instruction corresponding to the request data, the data processing operation performed on the logical entity object includes the following sub-operations: s601, S602, and S603.
In sub-operation S601, a target candidate SQL statement matching the type is determined based on the type of the data processing operation instruction.
The types mentioned above include, for example: select/update/delete/insert types, etc.
In sub-operation S602, the attribute values of the logical entity object and the corresponding data processing operation instruction are transmitted to the target candidate SQL statement, and a logical operation SQL statement is generated.
The logical entity object is an operation object, and the attribute value of the data processing operation instruction includes an attribute value of an operation condition. For example, a logical operation SQL statement generated based on a logical entity object and a corresponding data processing operation instruction: select id from table _ abc where id >12. Wherein "id >12" is an operating condition, 12 is an attribute value of a data processing operation instruction; "select id from world id >? ", operands and attribute values are default, and by passing in" table _ abc "and" 12", a logical operation SQL statement results.
In sub-operation S603, a data processing operation is performed on the logical entity object based on the logical operation SQL statement.
Fig. 7 schematically shows a detailed implementation flowchart of operation S204 according to an embodiment of the present disclosure.
According to an embodiment of the present disclosure, referring to fig. 7, in the operation S204, converting the data processing operation for the logical entity object into an operation for a physical object in a database to be implemented includes the following sub-operations: s701 and S702.
In sub-operation S701, the logical entity object of the logical operation SQL statement is converted into a physical object, and a physical operation SQL statement is generated.
After morpheme (token) analysis is carried out on the logical operation SQL statement, the logical entity object is replaced by a physical object, namely, the id (logical entity object) in the selection/id/table _ abc/where/expression id, >,10 "is replaced by the id _ a (physical object), and the table _ abc (logical entity object) is replaced by the def (physical object), so that the physical operation SQL statement is generated: "select id _ a from def where id >12". Here, "/" is used to divide the morphemes.
In sub-operation S702, a corresponding operation is performed on the physical object in the database based on the physical operation SQL statement.
Based on the above physical operation SQL statement: "select id _ a from def where id >12", this operation is performed for the id _ a column in the def table in the database, for the query condition "id >12".
A second exemplary embodiment of the present disclosure provides an apparatus for data processing.
Fig. 8 schematically shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure.
Referring to fig. 8, an apparatus 800 for data processing provided by an embodiment of the present disclosure includes: a receiving module 801, a parsing module 802, a logical operation module 803, and a physical operation module 804.
The receiving module 801 is configured to receive request data to be processed and a corresponding data processing operation instruction.
The parsing module 802 is configured to parse the request data into a logical entity object in the metadata field.
The logical operation module 803 is configured to perform a data processing operation on the logical entity object in the metadata domain according to a data processing operation instruction corresponding to the request data. The above logical operation module 803 includes a functional module or sub-module for implementing the sub-operations S2031 to S2032, or includes a functional module or sub-module for implementing the sub-operations S601 to S603.
The physical operation module 804 is configured to convert the data processing operation for the logical entity object into an operation for a physical object in a database, and implement the operation to obtain a processing result of the request data. The physical operation module 804 may include a functional module or a sub-module for implementing the sub-operations S2041 and S2042, or a functional module or a sub-module for implementing the sub-operations S701 and S702.
According to an embodiment of the present disclosure, the apparatus 800 further includes a pre-compiling module in addition to the receiving module 801, the parsing module 802, the logical operation module 803, and the physical operation module 804.
The pre-compiling module is used for pre-compiling candidate SQL statements, and the candidate SQL statements are used for database operation.
Alternatively, in another embodiment, the apparatus 800 includes the receiving module 801, the parsing module 802, the logical operation module 803, and the physical operation module 804, and stores the pre-compiled candidate SQL statements.
Any of the receiving module 801, the parsing module 802, the logical operation module 803, and the physical operation module 804 may be combined into one module to be implemented, or any one of the modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. At least one of the receiving module 801, the parsing module 802, the logical operation module 803 and the physical operation module 804 may be at least partially implemented as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware by any other reasonable manner of integrating or packaging a circuit, or may be implemented in any one of three implementations of software, hardware and firmware, or in a suitable combination of any of them. Alternatively, at least one of the receiving module 801, the parsing module 802, the logical operation module 803 and the physical operation module 804 may be at least partially implemented as a computer program module, which when executed may perform a corresponding function.
A third exemplary embodiment of the present disclosure provides an electronic apparatus.
Fig. 9 schematically shows a block diagram of an electronic device provided in an embodiment of the present disclosure.
Referring to fig. 9, an electronic device 900 provided in the embodiment of the present disclosure includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, where the processor 901, the communication interface 902, and the memory 903 complete communication with each other through the communication bus 904; a memory 903 for storing computer programs; the processor 901 is configured to implement the data processing method described above when executing the program stored in the memory.
A fourth exemplary embodiment of the present disclosure also provides a computer-readable storage medium. The above-mentioned computer-readable storage medium has stored thereon a computer program which, when executed by a processor, implements the method of data processing as described above.
The computer-readable storage medium may be contained in the apparatus/device described in the above embodiments; or may be separate and not incorporated into the device/apparatus. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
The previous description is only for the purpose of describing particular embodiments of the present disclosure, so as to enable those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (11)
1. A method of data processing, comprising:
receiving request data to be processed and a corresponding data processing operation instruction;
analyzing the request data into a logic entity object in a metadata field;
in the metadata field, carrying out data processing operation aiming at the logic entity object according to a data processing operation instruction corresponding to the request data; and
and converting the data processing operation aiming at the logic entity object into an operation aiming at a physical object in a database for implementation to obtain the processing result of the request data.
2. The method according to claim 1, wherein the metadata domain stores a preconfigured mapping relationship, and the mapping relationship is used for characterizing an association between a logical entity object in the logical table and a physical object in the physical table;
the converting the data processing operation aiming at the logic entity object into the operation aiming at the physical object in the database for implementation comprises the following steps:
converting the data operation aiming at the logic entity object into a target operation aiming at a physical object in a database based on the mapping relation; and
and implementing corresponding target operation aiming at the physical object in a database.
3. The method of claim 2, wherein converting the data operation for the logical entity object to a target operation for a physical object in a database based on the mapping relationship comprises:
determining a target position in a target physical table associated with the logical table position where the logical entity object is located based on the mapping relation; and
converting the data operation for the logical entity object to an operation to be performed for a physical object at the target location of the target physical table.
4. The method of claim 1, wherein parsing the request data into logical entity objects of metadata fields comprises:
parsing the request data into logical entity objects of a metadata realm based on an interceptor or a filter before the request data is processed.
5. The method according to claim 1, wherein in the metadata domain, performing a data processing operation on the logical entity object according to a data processing operation instruction corresponding to the request data includes:
correspondingly distributing the logic entity object obtained after the request data is analyzed to an execution instruction of the request data; and
and in the process that the request data is executed according to the execution instruction, carrying out data processing operation aiming at the logic entity object in the metadata field according to the data processing operation instruction corresponding to the request data.
6. The method according to claim 1, wherein the request data is associated with a user identifier, and data processing operations of different users are isolated based on the user identifier;
parsing the request data into a logical entity object in a metadata domain, comprising:
in the metadata domain, the request data is parsed into: a logical entity object associated with the user identification;
according to the data processing operation instruction corresponding to the request data, carrying out data processing operation on the logic entity object, including:
according to a data processing operation instruction corresponding to the request data, carrying out data processing operation on the logic entity object associated with the user identifier;
converting the data processing operation aiming at the logic entity object into an operation aiming at a physical object in a database for implementation, wherein the operation comprises the following steps:
and converting the data processing operation aiming at the logic entity object associated with the user identification into the operation aiming at the physical object associated with the user identification in the database for implementation.
7. The method of claim 1, further comprising:
precompiling candidate SQL sentences used for database operation; attribute values of the operands and/or operating conditions of the candidate SQL statement are either passed in when performing data processing operations on the requested data or passed in when performing operations on the physical object.
8. The method of claim 7,
the performing data processing operation on the logical entity object according to the data processing operation instruction corresponding to the request data includes:
determining a target candidate SQL statement matched with the type based on the type of the data processing operation instruction;
transmitting the attribute values of the logic entity object and the corresponding data processing operation instruction into the target candidate SQL statement to generate a logic operation SQL statement; and
performing data processing operations on the logical entity object based on the logical operation SQL statement;
the converting the data processing operation aiming at the logic entity object into the operation aiming at the physical object in the database for implementation comprises the following steps:
converting the logic entity object of the logic operation SQL statement into a physical object to generate a physical operation SQL statement; and
and based on the physical operation SQL statement, corresponding operation is carried out aiming at the physical object in the database.
9. An apparatus for data processing, comprising:
the receiving module is used for receiving the request data to be processed and the corresponding data processing operation instruction;
the analysis module is used for analyzing the request data into a logic entity object in the metadata field;
the logic operation module is used for implementing the data processing operation aiming at the logic entity object in the metadata field according to a data processing operation instruction corresponding to the request data; and
and the physical operation module is used for converting the data processing operation aiming at the logic entity object into the operation aiming at the physical object in the database to implement so as to obtain the processing result of the request data.
10. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
a memory for storing a computer program;
a processor for implementing the method of any one of claims 1 to 8 when executing a program stored on a memory.
11. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110919758.5A CN115905274A (en) | 2021-08-11 | 2021-08-11 | Data processing method and device, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110919758.5A CN115905274A (en) | 2021-08-11 | 2021-08-11 | Data processing method and device, electronic equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115905274A true CN115905274A (en) | 2023-04-04 |
Family
ID=86476570
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110919758.5A Pending CN115905274A (en) | 2021-08-11 | 2021-08-11 | Data processing method and device, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115905274A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116680299A (en) * | 2023-06-05 | 2023-09-01 | 粤港澳大湾区(广东)国创中心 | Database query method, system, storage medium and device |
-
2021
- 2021-08-11 CN CN202110919758.5A patent/CN115905274A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116680299A (en) * | 2023-06-05 | 2023-09-01 | 粤港澳大湾区(广东)国创中心 | Database query method, system, storage medium and device |
| CN116680299B (en) * | 2023-06-05 | 2024-04-09 | 粤港澳大湾区(广东)国创中心 | Database query method, system, storage medium and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11163739B2 (en) | Database table format conversion based on user data access patterns in a networked computing environment | |
| US9996565B2 (en) | Managing an index of a table of a database | |
| CN102521230B (en) | For the result type that data with good conditionsi show | |
| US10180984B2 (en) | Pivot facets for text mining and search | |
| US9418101B2 (en) | Query optimization | |
| US10002170B2 (en) | Managing a table of a database | |
| US10303689B2 (en) | Answering natural language table queries through semantic table representation | |
| US20150331675A1 (en) | Modeling representational state transfer application programming interfaces | |
| KR101122629B1 (en) | Method for creation of xml document using data converting of database | |
| US10120886B2 (en) | Database integration of originally decoupled components | |
| US11030242B1 (en) | Indexing and querying semi-structured documents using a key-value store | |
| US20150363435A1 (en) | Declarative Virtual Data Model Management | |
| US20150012553A1 (en) | Dynamic assignment of business logic based on schema mapping metadata | |
| CN111708805A (en) | Data query method and device, electronic equipment and storage medium | |
| CN110110184B (en) | Information inquiry method, system, computer system and storage medium | |
| Kim et al. | Comprehensive Knowledge Archive Network harvester improvement for efficient open‐data collection and management | |
| CN115905274A (en) | Data processing method and device, electronic equipment and medium | |
| CN113760948A (en) | Data query method and device | |
| CN115705435A (en) | Data processing method, device, equipment and medium for cloud service | |
| CN115905415A (en) | Database operation method and device, electronic equipment and medium | |
| KR20130031946A (en) | System for providing trend information of application and method thereof | |
| US20230247111A1 (en) | Runtime module conversion | |
| WO2023164294A1 (en) | Query splitter for an inverted index datastore | |
| US10819784B1 (en) | API mediation for online services | |
| US20200320162A1 (en) | Management of content objects for ingestion by multiple entities |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |