CN113742334A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN113742334A CN113742334A CN202010603593.6A CN202010603593A CN113742334A CN 113742334 A CN113742334 A CN 113742334A CN 202010603593 A CN202010603593 A CN 202010603593A CN 113742334 A CN113742334 A CN 113742334A
- Authority
- CN
- China
- Prior art keywords
- service
- data
- field
- value
- data table
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 21
- 238000000034 method Methods 0.000 claims abstract description 24
- 230000008859 change Effects 0.000 claims description 26
- 238000012545 processing Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 description 13
- 230000004048 modification Effects 0.000 description 10
- 238000012986 modification Methods 0.000 description 10
- 230000001360 synchronised effect Effects 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data processing method and device, and relates to the technical field of computers. The method comprises the following steps: after a data marking request is received, determining a unique identifier of service data to be processed and a service field to be marked in a first data table; one row in the first data table is used for storing a piece of service data, and the service data comprises values of all service fields and service identification field values used for marking the service fields; generating a first service identification value represented in a binary form according to the service field to be marked, and converting the first service identification value into a second service identification value represented in a decimal form; and updating the service identification field value in the service data corresponding to the unique identification in the first data table according to the second service identification value. Through the steps, the individualized data marking requirement of the service system can be met, and the identification storage cost and the loading cost of the service data are reduced.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data processing method and apparatus.
Background
As internet technology has developed, people increasingly rely on various network service platforms, such as e-commerce platforms, social platforms, and so on. A web services platform typically includes a plurality of business systems. Under different service scenes, each service system often has individualized and customized data requirements. In order to meet the personalized and customized data requirements of each business system, an identifier (marking) is often required to be set for business data. For example, in an e-commerce platform, a scenario that one main data supports multiple service systems often exists, and a service system has a customized data requirement, for example, part of fields in a service data table corresponding to the service system need to be synchronized with the main data for changing, and part of fields need to be personalized for maintenance, which requires setting an identifier such as "mark" for indicating whether each field in the service data table needs to be updated with the main data.
In the prior art, data marking is mainly performed in a mode of adding a service identification table in a database. And one line in the service identification table is used for marking one piece of service data in the original service data table.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art: with the increasing demand of marking service data, the storage capacity of the identification data is continuously enlarged; under the scene of numerous service data, the problems of high storage cost of the identification and high subsequent data loading cost exist by adopting the conventional data marking method.
Disclosure of Invention
In view of this, the present invention provides a data processing method and apparatus, which can meet the personalized data marking requirement of a service system, and reduce the identifier storage cost and the loading cost of service data.
To achieve the above object, according to one aspect of the present invention, a data processing method is provided.
The data processing method of the invention comprises the following steps: after a data marking request is received, determining a unique identifier of service data to be processed and a service field to be marked in a first data table; one row in the first data table is used for storing a piece of service data, and the service data comprises values of all service fields and service identification field values used for marking the service fields; generating a first service identification value represented in a binary form according to the service field to be marked, and converting the first service identification value into a second service identification value represented in a decimal form; and updating the service identification field value in the service data corresponding to the unique identification in the first data table according to the second service identification value.
Optionally, the generating a first service identifier value represented in a binary form according to the service field to be marked includes: acquiring a service identifier field value in the service data corresponding to the unique identifier in a first data table; converting the acquired field value of the service identifier into a binary identifier array; inquiring configuration information according to the service field to be marked so as to determine the position of the service field to be marked in the binary identification array; and updating the value of the position of the service field to be marked in the binary identification array to obtain the first service identification value.
Optionally, the updating, according to the second service identification value, a service identification field value in the service data corresponding to the unique identification in the first data table includes: and when updating operation is carried out according to the second service identification value, updating the service identification field value in the service data corresponding to the unique identification in the first data table in an optimistic locking mode.
Optionally, the service identification field is used to indicate whether each service field in the first data table needs to be updated together with the second data table.
Optionally, the updating the value of the position where the service field to be marked is located in the binary identification array includes: when the service field to be marked is a field which needs to be updated together with a second data table, updating the value of the position of the service field to be marked in the binary identification array to be a first Boolean value; and when the service field to be marked is a field which does not need to be updated together with a second data table, updating the value of the position of the service field to be marked in the binary identification array to be a second Boolean value.
Optionally, the method further comprises: after receiving the second data table change message, determining the change details of the second data table; and updating the value of the service field which needs to be updated together with the second data table in the first data table according to the change details of the second data table.
Optionally, the change details of the second data table include: changing a unique identifier of data and changing a service field value in the data; the updating the value of the service field which needs to be updated together with the second data table in the first data table according to the change details of the second data table includes: assembling a first data object according to the unique identification of the changed data and the service field value in the changed data; acquiring a service identifier field value in the service data corresponding to the unique identifier of the changed data in the first data table; converting the obtained service identification field value into a fifth service identification value represented in a binary form, and determining a service field which needs to be updated together with the second data table in the first data table according to the fifth service identification value; assembling a second data object according to the unique identification of the changed data and the service field which needs to be updated together with a second data table in the first data table; assigning values to the second data object according to the first data object; and updating the first data table according to the assigned second data object.
To achieve the above object, according to another aspect of the present invention, there is provided a data processing apparatus.
The data processing apparatus of the present invention includes: the determining module is used for determining the unique identifier of the service data to be processed and the service field to be marked in the first data table after the data marking request is received; one row in the first data table is used for storing a piece of service data, and the service data comprises values of all service fields and service identification field values used for marking the service fields; the generating module is used for generating a first service identification value expressed in a binary form according to the service field to be marked and converting the first service identification value into a second service identification value expressed in a decimal form; and the updating module is used for updating the service identification field value in the service data corresponding to the unique identification in the first data table according to the second service identification value.
To achieve the above object, according to still another aspect of the present invention, there is provided an electronic apparatus.
The electronic device of the present invention includes: one or more processors; and storage means for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the data processing method of the present invention.
To achieve the above object, according to still another aspect of the present invention, there is provided a computer-readable medium.
The computer-readable medium of the present invention has stored thereon a computer program which, when executed by a processor, implements the data processing method of the present invention.
One embodiment of the above invention has the following advantages or benefits: the method comprises the steps of determining the unique identification of the service data to be processed in a first data table and the service field to be marked after a data marking request is received, generating a first service identification value expressed in a binary form according to the service field to be marked, converting the first service identification value into a second service identification value expressed in a decimal form, and updating the service identification field value in the service data corresponding to the unique identification in the first data table according to the second service identification value, so that the individualized data marking requirement of a service system can be met, and the identification storage cost and the loading cost of the service data are reduced.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
fig. 1 is a main flow chart diagram of a data processing method according to a first embodiment of the present invention;
FIG. 2 is a schematic main flow chart of a data processing method according to a second embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a process of synchronous update of service fields in a data processing method according to a third embodiment of the present invention;
FIG. 4 is a schematic diagram of the main blocks of a data processing apparatus according to a fourth embodiment of the present invention;
FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
FIG. 6 is a schematic block diagram of a computer system suitable for use with the electronic device to implement an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiments and technical features of the embodiments of the present invention may be combined with each other without affecting the implementation of the present invention.
First embodiment
Fig. 1 is a main flow diagram of a data processing method according to a first embodiment of the present invention. As shown in fig. 1, the data processing method according to the embodiment of the present invention includes:
step S101: and after the data marking request is received, determining the unique identifier of the service data to be processed and the service field to be marked in the first data table.
The data marking request may also be referred to as a "service data marking request" or a "request for setting a service identifier for service data", or the like.
In an optional example, the data marking request may carry a unique identifier of the service data to be processed in the first data table and a service field to be marked. In this optional example, the unique identifier of the to-be-processed service data in the first data table carried by the data marking request and the service field to be marked may be obtained from the data marking request. For example, a certain data marking request includes the following request parameters { id:1, syncField: wartatus }, where "id: 1" indicates that the unique identifier of the service data to be processed in the first data table (for example, it is specifically a self-increment key) is "1", "syncField: the "wartus" indicates that the traffic field to be marked in the first data table is "wartus".
In another optional example, the data marking request may not carry the unique identifier of the service data to be processed in the first data table and the service field to be marked. In this optional example, the unique identifier of the to-be-processed service data and the to-be-marked service field corresponding to the unique identifier can be queried according to the information carried in the data marking request.
And one row in the first data table is used for storing a piece of service data, and the service data comprises values of all service fields and service identification field values used for marking the service fields. For example, one piece of service data in the first data table includes the value of the service field a, the value of the service field B, the value of the service field C, and the value of the service identification field "mark" for marking the service field A, B, C.
Further, in order to save storage space, the data type used by the service identification field value in the first data table is a decimal numeric type, such as bigint, or int, etc. bigint is an integer data type supported by a common relational database, and the occupied storage space of the bigint is generally 8 bytes; int is an integer data type that occupies a memory space of typically 4 bytes. After decimal conversion binary processing is carried out on the service identification field value in the memory, the service identification field value expressed in a binary form has the following meanings: a service field in the first data table corresponds to a position in the service identifier field value represented in the binary form, and the value of the position represents a mark value of the service field, specifically a first boolean value or a second boolean value. In specific implementation, if the first boolean value is 1, the second boolean value is 0; if the first Boolean value is set to 0, the second Boolean value is set to 1. For example, assuming that the service identifier field value represented in the binary form is 01011, and there are five service fields a to E in the first data table, the value of the service identifier field value at the first position from left to right represents the tag value of the service field a, the value of the service field B at the second position represents the tag value of the service field B, the value of the service field C at the third position represents the tag value of the service field D at the fourth position, and the value of the service field E at the fifth position represents the tag value of the service field E.
Step S102: and generating a first service identification value represented in a binary form according to the service field to be marked.
The service field to be marked may be specifically a service field to be marked as a first boolean value, and/or a service field to be marked as a second boolean value. Under different service scenes, the mark values of the service fields have different actual meanings. Exemplarily, in a certain service scenario, the value of the flag of the service field is a first boolean value indicating that the service field needs to be updated together with the primary data, and the value of the flag of the service field is a second boolean value indicating that the service field does not need to be updated together with the primary data. Furthermore, in this example, the service field to be marked may be a service field that needs to be updated together with the main data, and/or a service field that does not need to be updated together with the main data.
Step S103: and converting the first service identification value into a second service identification value expressed in a decimal form.
Step S104: and updating the service identification field value in the service data corresponding to the unique identification in the first data table according to the second service identification value.
Further, after obtaining the second service identification value expressed in decimal, in order to prevent data confusion caused by concurrent modification, the service identification field value in the service data corresponding to the unique identification in the first data table can be updated in an optimistic lock manner. In an optional implementation manner, the update of the service identification field value may be performed according to the unique identification of the service data to be processed carried by the data marking request and the original service identification field value in the first data table as conditions. For example, the following database update statements may be employed: update ware set mark [ # newmark where id [ # id ] and mark [ # oldmark ]. Wherein, "# newmark" represents the second service identification value, "# id" represents the unique identification of the service data to be processed, and "# oldmark" represents the service field identification value in the database before modification.
In the embodiment of the invention, after a data marking request is received, the unique identifier of the service data to be processed in the first data table and the service field to be marked are determined, the first service identifier value expressed in a binary form is generated according to the service field to be marked, the first service identifier value is converted into the second service identifier value expressed in a decimal form, and the service identifier field value in the service data corresponding to the unique identifier in the first data table is updated according to the second service identifier value.
Second embodiment
Fig. 2 is a main flow chart of a data processing method according to a second embodiment of the present invention. As shown in fig. 2, the data processing method according to the embodiment of the present invention includes:
step S201: and receiving a data marking request.
Step S202: and determining the unique identifier of the service data to be processed and the service field to be marked in the first data table according to the data marking request.
In an optional example, the data marking request may carry a unique identifier of the service data to be processed in the first data table and a service field to be marked. In this optional example, the unique identifier of the to-be-processed service data in the first data table carried by the data marking request and the service field to be marked may be obtained from the data marking request. For example, a certain data marking request includes the following request parameters { id:1, syncField: wartatus }, where "id: 1" indicates that the unique identifier of the service data to be processed in the first data table (for example, it is specifically a self-increment key) is "1", "syncField: the "wartus" indicates that the traffic field to be marked in the first data table is "wartus".
And one row in the first data table is used for storing a piece of service data, and the service data comprises values of all service fields and service identification field values used for marking the service fields. For example, one piece of service data in the first data table includes the value of the service field a, the value of the service field B, the value of the service field C, and the value of the service identification field "mark" for marking the service field A, B, C.
Further, in order to save storage space, the data type used by the service identification field value in the first data table is a decimal numeric type, such as bigint, or int, etc. bigint is an integer data type supported by a common relational database, and the occupied storage space of the bigint is generally 8 bytes; int is an integer data type that occupies a memory space of typically 4 bytes.
Step S203: and acquiring a service identification field value in the service data corresponding to the unique identification in the first data table.
In this step, the first data table may be queried according to the unique identifier of the service data to be processed, so as to obtain the value of the service identifier field in the service data corresponding to the value. For example, assuming that the unique identifier of the service data to be processed satisfies "id ═ 1", and the service identifier field is denoted by "mark", the following database query statement may be employed: select id, mark from ware where id is 1.
Step S204: and converting the acquired business identifier field value into a binary identifier array.
In this step, a decimal to binary operation may be performed on the service identifier field value obtained from the first data table in the memory to obtain a service identifier field value represented in a binary form, specifically, a binary identifier array. And a service field in the first data table corresponds to a position in the binary identification array, and the value of the position represents a mark value of the service field, specifically a first boolean value or a second boolean value. In specific implementation, if the first boolean value is 1, the second boolean value is 0; if the first Boolean value is set to 0, the second Boolean value is set to 1.
Step S205: and inquiring configuration information according to the service field to be marked so as to determine the position of the service field to be marked in the binary identification array.
In the embodiment of the present invention, the corresponding relationship between each service field in the first data table and the specific position in the binary identifier array may be configured in advance. For example, assuming that there are five service fields a to E in the first data table, and the binary identification array includes five positions in total, the following information may be configured: the first position in the binary number group from left to right corresponds to a service field A, the second position corresponds to a service field B, the third position corresponds to a service field C, the fourth position corresponds to a service field D, and the fifth position corresponds to a service field E. Further, assuming that the service field to be marked is the service field C, the third position of the service field in the binary identification array can be determined by querying the configuration information.
Step S206: and updating the value of the position of the service field to be marked in the binary identification array to obtain a first service identification value represented in a binary form.
The service field to be marked may be specifically a service field to be marked as a first boolean value, and/or a service field to be marked as a second boolean value. Under different service scenes, the mark values of the service fields have different actual meanings. Illustratively, in a certain service scenario, the value of the flag of the service field is the first boolean value indicating that the service field needs to be updated together with the second data table, and the value of the flag of the service field is the second boolean value indicating that the service field does not need to be updated together with the second data table. Furthermore, in this example, the service field to be marked may be a service field that needs to be updated together with the second data table, and/or a service field that does not need to be updated together with the second data table.
Further, when the service field to be marked is specifically a service field that needs to be updated together with the second data table, step S206 specifically includes: and updating the value of the position of the service field to be marked in the binary identification array to be a first Boolean value. For example, assuming that the first boolean value is 1, the second boolean value is 0, the service field to be marked is the service field C that needs to be updated synchronously, and the position of the service field C in the binary identification array "01011" is the third bit from left to right, then "0" of the third bit is replaced by 1, so as to obtain the first service identification value "01111".
When the service field to be marked is specifically a service field that needs to be updated together with the main data and a field that does not need to be updated together with the second data table, step S206 specifically includes: when the service field to be marked is a field which needs to be updated together with a second data table, updating the value of the position of the service field to be marked in the binary identification array to be a first Boolean value; and when the service field to be marked is a field which does not need to be updated together with a second data table, updating the value of the position of the service field to be marked in the binary identification array to be a second Boolean value. For example, assuming that the first boolean value is 1, the second boolean value is 0, the service fields to be marked are a service field C requiring synchronous update and a field B not requiring synchronous update, and the positions of the service field B, C in the binary identification array "01011" are the second and third bits from left to right, respectively, then "1" of the second bit is replaced by 0, and "0" of the third bit is replaced by 1, so as to obtain the first service identification value "00111".
In the embodiment of the present invention, steps S203 to S206 are an optional implementation of step S202 in the embodiment shown in fig. 1. In specific implementation, step S102 in the embodiment shown in fig. 1 may also take other alternative implementations.
Step S207: converting the first service identification value into a second service identification value expressed in decimal form.
In this step, a second service identification value can be obtained by performing binary to decimal operation on the first service identification value.
In the embodiment of the invention, the service identification field value is stored in the database table in a decimal form, and decimal-to-binary operation is carried out in the memory to obtain the service identification value in a binary form, so that the identification of the service data is convenient to modify, the storage space required by the identification of the service data stored in the database is saved, and the identification storage cost and the loading cost of the service data are reduced.
Step S208: and updating the service identification field value in the service data corresponding to the unique identification in the first data table according to the second service identification value.
Further, after obtaining the second service identification value expressed in decimal, in order to prevent data confusion caused by concurrent modification, the service identification field value in the service data corresponding to the unique identification in the first data table can be updated in an optimistic lock manner. In an optional implementation manner, the update of the service identification field value may be performed according to the unique identification of the service data to be processed carried by the data marking request and the original service identification field value in the first data table as conditions. For example, the following database update statements may be employed: update ware set mark [ # newmark where id [ # id ] and mark [ # oldmark ]. Wherein, "# newmark" represents the second service identification value, "# id" represents the unique identification of the service data to be processed, and "# oldmark" represents the service field identification value in the database before modification.
In the embodiment of the invention, the individualized data marking requirement of the service system can be met through the steps, and the identification storage cost and the loading cost of the service data are reduced.
Third embodiment
The service scene applicable to the data processing method of the embodiment of the invention is described as follows: due to the development of services in an e-commerce platform, a scenario that main data supports other services often exists, because the existing service scenario has a customized data requirement, part of service fields stored in a first data table need to be synchronized with main data (specifically, data in a second data table) to be changed, and part of service fields need to be personalized maintained. In the service scene, the fact that the service field is marked to take the value of the first Boolean indicates that the service field needs to be updated together with the second data table, and the fact that the service field is marked to take the value of the second Boolean indicates that the service field does not need to be updated together with the second data table. In addition to the data marking process shown in fig. 2, the data processing method according to the embodiment of the present invention further includes a service field synchronous updating process.
Fig. 3 is a schematic diagram illustrating a flow of synchronous update of service fields in a data processing method according to a third embodiment of the present invention. As shown in fig. 3, the service field synchronous update process includes:
step S301: upon receiving the second data table change message, determining the change details of the second data table.
In an optional example, the second data table change message carries change details of the second data table. In this alternative example, second data table change details may be obtained from the second data table change message.
Wherein the change details of the second data table may include: a unique identification of the change data, and a value of a service field in the change data. For example, a certain second data table change message includes the following parameters: { id:1, wartstatus: 1 }. Wherein, "id: 1 ' represents that the unique identifier of the changed data is ' 1 ', and ' wartstatus: 1 ' represents that the value of a service field wartstatus in the changed data is 1.
In another optional example, the second data table change message does not carry change details of the second data table. In this optional example, the change details of the second data table may be queried according to the related parameters carried by the second data table change message.
Step S302: and updating the value of the service field which needs to be updated together with the second data table in the first data table according to the change details of the second data table.
In an optional example, step S302 specifically includes: step a1 to step a 5.
Step a 1: and assembling a first data object according to the unique identification of the changed data and the service field value in the changed data.
Illustratively, the first data object may be a map object. A map object is an object that maps keys to values. In step a1, the first data object may be constructed by uniquely identifying the changed data as a key (key) of the map object and the value of the business field in the changed data as a value (value) of the map object.
Step a 2: and acquiring a service identification field value in the service data corresponding to the unique identification of the changed data in the first data table.
In this step, the first data table may be queried according to the unique identifier of the changed data to obtain a value of a service identifier field in the service data corresponding to the unique identifier of the changed data. For example, assuming that the change data unique identifier satisfies "id ═ 2", and the service identifier field is denoted by "mark", the following database query statement may be employed: select id, mark from ware where id is 2.
Step a 3: and converting the obtained service identification field value into a fifth service identification value represented in a binary form, and determining a service field which needs to be updated together with the second data table in the first data table according to the fifth service identification value.
In this step, a decimal to binary operation may be performed on the service identifier field value obtained from the first data table in the memory to obtain a fifth service identifier value represented in a binary form. After the fifth service identification value is obtained, positions of the fifth service identification value, where all values are the first boolean value (for example, 1), may be traversed, and the service fields corresponding to these positions may be determined by querying the configuration information, so that it may be determined which service fields in the first data table need to be updated together with the second data table. For example, assuming that the fifth identification value is "01110", the positions of the fifth identification value with the median value of 1 are the second bit, the third bit, and the fourth bit, respectively, it can be known through querying the configuration information that the service fields sequentially corresponding to these positions are the service field B, the service field C, and the service field D, and thus it can be determined that the service fields that need to be updated together with the second data table in the first data table are the service field B, the service field C, and the service field D.
Step a 4: and assembling a second data object according to the unique identification of the changed data and the service field which needs to be updated together with the second data table in the first data table.
In this step, the second data object may be constructed by using the unique identifier of the changed data as a key (key) and using the value (value) of the service field in the first data table that needs to be updated together with the second data table.
Step a 5: assigning values to the second data object according to the first data object; and updating the first data table according to the assigned second data object.
In an optional embodiment, said assigning the second data object according to the first data object includes: for each service field in the first data object, judging whether the service field exists in the second data object; if yes, assigning the value of the service field to a second data object; if not, traversing the next service field in the first data object until all the service fields in the first data object are traversed.
Further, after obtaining the assigned second data object, in order to prevent data confusion caused by concurrent modification, the service field value in the first data table may be updated in an optimistic lock manner.
In the embodiment of the invention, the synchronous updating process of the service fields is realized through the steps, the individualized data requirement of the service system is met, and meanwhile, the storage space of the service data identification is saved.
Fourth embodiment
Fig. 4 is a schematic diagram of main blocks of a data processing apparatus according to a fourth embodiment of the present invention. As shown in fig. 4, a data processing apparatus 400 according to an embodiment of the present invention includes: a determining module 401, a generating module 402, and an updating module 403.
The determining module 401 is configured to determine, after receiving the data marking request, a unique identifier of the to-be-processed service data in the first data table and a to-be-marked service field.
In an optional example, the data marking request may carry a unique identifier of the service data to be processed in the first data table and a service field to be marked. In this optional example, the determining module 401 may obtain, from the data marking request, a unique identifier of the to-be-processed service data in the first data table carried by the data marking request, and a service field to be marked. For example, a certain data marking request includes the following request parameters { id:1, syncField: wartatus }, where "id: 1" indicates that the unique identifier of the service data to be processed in the first data table (for example, it is specifically a self-increment key) is "1", "syncField: the "wartus" indicates that the traffic field to be marked in the first data table is "wartus".
In another optional example, the data marking request may not carry the unique identifier of the service data to be processed in the first data table and the service field to be marked. In this optional example, the determining module 401 may query, according to the information carried in the data marking request, a unique identifier of the to-be-processed service data corresponding to the information, and a service field to be marked.
And one row in the first data table is used for storing a piece of service data, and the service data comprises values of all service fields and service identification field values used for marking the service fields. For example, one piece of service data in the first data table includes the value of the service field a, the value of the service field B, the value of the service field C, and the value of the service identification field "mark" for marking the service field A, B, C.
Further, in order to save storage space, the data type used by the service identification field value in the first data table is a decimal numeric type, such as bigint, or int, etc. bigint is an integer data type supported by a common relational database, and the occupied storage space of the bigint is generally 8 bytes; int is an integer data type that occupies a memory space of typically 4 bytes. After decimal conversion binary processing is carried out on the service identification field value in the memory, the service identification field value expressed in a binary form has the following meanings: a service field in the first data table corresponds to a position in the service identifier field value represented in the binary form, and the value of the position represents a mark value of the service field, specifically a first boolean value or a second boolean value. In specific implementation, if the first boolean value is 1, the second boolean value is 0; if the first Boolean value is set to 0, the second Boolean value is set to 1. For example, assuming that the service identifier field value represented in the binary form is 01011, and there are five service fields a to E in the first data table, the value of the service identifier field value at the first position from left to right represents the tag value of the service field a, the value of the service field B at the second position represents the tag value of the service field B, the value of the service field C at the third position represents the tag value of the service field D at the fourth position, and the value of the service field E at the fifth position represents the tag value of the service field E.
A generating module 402, configured to generate a first service identification value represented in a binary form according to the service field to be marked, and convert the first service identification value into a second service identification value represented in a decimal form.
The service field to be marked may be specifically a service field to be marked as a first boolean value, and/or a service field to be marked as a second boolean value. Under different service scenes, the mark values of the service fields have different actual meanings. Exemplarily, in a certain service scenario, the value of the flag of the service field is a first boolean value indicating that the service field needs to be updated together with the primary data, and the value of the flag of the service field is a second boolean value indicating that the service field does not need to be updated together with the primary data. Furthermore, in this example, the service field to be marked may be a service field that needs to be updated together with the main data, and/or a service field that does not need to be updated together with the main data.
In an optional example, the generating module 402 generates the first service identification value represented in binary form according to the service field to be marked includes: the generating module 402 obtains a service identifier field value in the service data corresponding to the unique identifier in the first data table; the generating module 402 converts the acquired value of the service identifier field into a binary identifier array; the generating module 402 queries configuration information according to the service field to be marked to determine the position of the service field to be marked in the binary identification array; the generating module 402 updates the value of the position of the service field to be marked in the binary identification array to obtain the first service identification value.
An updating module 403, configured to update, according to the second service identifier, a service identifier field value in the service data corresponding to the unique identifier in the first data table.
In an alternative example, after obtaining the second service identification value expressed in decimal, in order to prevent data confusion caused by concurrent modification, the updating module 403 may update the service identification field value in the service data corresponding to the unique identification in the first data table in an optimistic lock manner. In an optional embodiment, the updating module 403 may update the service identifier field value according to a condition that the unique identifier of the service data to be processed is carried by the data marking request and the original service identifier field value in the first data table. For example, the following database update statements may be employed: update ware set mark [ # newmark where id [ # id ] and mark [ # oldmark ]. Wherein, "# newmark" represents the second service identification value, "# id" represents the unique identification of the service data to be processed, and "# oldmark" represents the service field identification value in the database before modification.
In the device of the embodiment of the invention, after a determining module receives a data marking request, a unique identifier of service data to be processed and a service field to be marked in a first data table are determined, a generating module generates a first service identifier value expressed in a binary form according to the service field to be marked, the first service identifier value is converted into a second service identifier value expressed in a decimal form, and an updating module updates a service identifier field value in the service data corresponding to the unique identifier in the first data table according to the second service identifier value, so that the individualized data marking requirement of a service system can be met, and the identifier storage cost and the loading cost of the service data are reduced.
Fig. 5 shows an exemplary system architecture 500 of a data processing method or data processing apparatus to which embodiments of the present invention may be applied.
As shown in fig. 5, the system architecture 500 may include terminal devices 501, 502, 503, a network 504, and a server 505. The network 504 serves to provide a medium for communication links between the terminal devices 501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 501, 502, 503 to interact with a server 505 over a network 504 to receive or send messages or the like. The terminal devices 501, 502, 503 may have various communication client applications installed thereon, such as a shopping application, a web browser application, a search application, an instant messaging tool, a mailbox client, social platform software, and the like.
The terminal devices 501, 502, 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 505 may be a server providing various services, such as a background management server providing support for data marking requests issued by users using the terminal devices 501, 502, 503. The background management server can perform data marking operation after receiving the data marking request, and return the data marking result to the terminal equipment.
It should be noted that the data processing method provided by the embodiment of the present invention may be executed by the server 505, and accordingly, the data processing apparatus is disposed in the server 505.
It should be understood that the number of terminal devices, networks, and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 6, shown is a block diagram of a computer system 600 suitable for use with the electronic device implementing an embodiment of the present invention. The computer system illustrated in FIG. 6 is only one example and should not impose any limitations on the scope of use or functionality of embodiments of the invention.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 601.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a determination module, a generation module, and an update module. The names of these modules do not in some cases constitute a limitation on the module itself, and for example, an update module may also be described as a "module that updates a first data table".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to perform the following: after a data marking request is received, determining a unique identifier of service data to be processed and a service field to be marked in a first data table; one row in the first data table is used for storing a piece of service data, and the service data comprises values of all service fields and service identification field values used for marking the service fields; generating a first service identification value represented in a binary form according to the service field to be marked, and converting the first service identification value into a second service identification value represented in a decimal form; and updating the service identification field value in the service data corresponding to the unique identification in the first data table according to the second service identification value.
According to the technical scheme of the embodiment of the invention, the individualized data marking requirement of the service system can be met, and the identification storage cost and the loading cost of the service data are reduced.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method of data processing, the method comprising:
after a data marking request is received, determining a unique identifier of service data to be processed and a service field to be marked in a first data table; one row in the first data table is used for storing a piece of service data, and the service data comprises values of all service fields and service identification field values used for marking the service fields;
generating a first service identification value represented in a binary form according to the service field to be marked, and converting the first service identification value into a second service identification value represented in a decimal form;
and updating the service identification field value in the service data corresponding to the unique identification in the first data table according to the second service identification value.
2. The method of claim 1, wherein the generating a first service identification value represented in binary form according to the service field to be marked comprises:
acquiring a service identifier field value in the service data corresponding to the unique identifier in a first data table; converting the acquired field value of the service identifier into a binary identifier array; inquiring configuration information according to the service field to be marked so as to determine the position of the service field to be marked in the binary identification array; and updating the value of the position of the service field to be marked in the binary identification array to obtain the first service identification value.
3. The method of claim 1, wherein updating the service identifier field value in the service data corresponding to the unique identifier in the first data table according to the second service identifier value comprises:
and when updating operation is carried out according to the second service identification value, updating the service identification field value in the service data corresponding to the unique identification in the first data table in an optimistic locking mode.
4. The method of claim 2, wherein the service identification field is used to indicate whether each service field in the first data table needs to be updated with the second data table.
5. The method according to claim 4, wherein the updating the value of the position of the service field to be marked in the binary identification array comprises:
when the service field to be marked is a field which needs to be updated together with a second data table, updating the value of the position of the service field to be marked in the binary identification array to be a first Boolean value; and when the service field to be marked is a field which does not need to be updated together with a second data table, updating the value of the position of the service field to be marked in the binary identification array to be a second Boolean value.
6. The method of claim 4, further comprising:
after receiving the second data table change message, determining the change details of the second data table; and updating the value of the service field which needs to be updated together with the second data table in the first data table according to the change details of the second data table.
7. The method of claim 5, wherein the change details of the second data table comprise: changing a unique identifier of data and changing a service field value in the data; the updating the value of the service field which needs to be updated together with the second data table in the first data table according to the change details of the second data table includes:
assembling a first data object according to the unique identification of the changed data and the service field value in the changed data; acquiring a service identifier field value in the service data corresponding to the unique identifier of the changed data in the first data table; converting the obtained service identification field value into a fifth service identification value represented in a binary form, and determining a service field which needs to be updated together with the second data table in the first data table according to the fifth service identification value; assembling a second data object according to the unique identification of the changed data and the service field which needs to be updated together with a second data table in the first data table; assigning values to the second data object according to the first data object; and updating the first data table according to the assigned second data object.
8. A data processing apparatus, characterized in that the apparatus comprises:
the determining module is used for determining the unique identifier of the service data to be processed and the service field to be marked in the first data table after the data marking request is received; one row in the first data table is used for storing a piece of service data, and the service data comprises values of all service fields and service identification field values used for marking the service fields;
the generating module is used for generating a first service identification value expressed in a binary form according to the service field to be marked and converting the first service identification value into a second service identification value expressed in a decimal form;
and the updating module is used for updating the service identification field value in the service data corresponding to the unique identification in the first data table according to the second service identification value.
9. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010603593.6A CN113742334A (en) | 2020-06-29 | 2020-06-29 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010603593.6A CN113742334A (en) | 2020-06-29 | 2020-06-29 | Data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113742334A true CN113742334A (en) | 2021-12-03 |
Family
ID=78728078
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010603593.6A Pending CN113742334A (en) | 2020-06-29 | 2020-06-29 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113742334A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1996308A (en) * | 2006-12-30 | 2007-07-11 | 陈�峰 | Object-orientation marking method |
| JP2009093569A (en) * | 2007-10-11 | 2009-04-30 | Hitachi Ltd | Business service using system, business service execution system, business service using method and program, and business service execution method and program |
| WO2017067253A1 (en) * | 2015-10-23 | 2017-04-27 | 华为技术有限公司 | Method and device for confirming traffic category-based indication |
| CN106897335A (en) * | 2016-06-28 | 2017-06-27 | 阿里巴巴集团控股有限公司 | A kind of storage method of business datum, the generation method of record identifier and device |
| CN109740120A (en) * | 2018-12-14 | 2019-05-10 | 深圳壹账通智能科技有限公司 | The generation method of unique identification, device, computer equipment under distributed environment |
| CN110928912A (en) * | 2018-09-20 | 2020-03-27 | 北京京东尚科信息技术有限公司 | Method and device for generating unique identifier |
-
2020
- 2020-06-29 CN CN202010603593.6A patent/CN113742334A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1996308A (en) * | 2006-12-30 | 2007-07-11 | 陈�峰 | Object-orientation marking method |
| JP2009093569A (en) * | 2007-10-11 | 2009-04-30 | Hitachi Ltd | Business service using system, business service execution system, business service using method and program, and business service execution method and program |
| WO2017067253A1 (en) * | 2015-10-23 | 2017-04-27 | 华为技术有限公司 | Method and device for confirming traffic category-based indication |
| CN106897335A (en) * | 2016-06-28 | 2017-06-27 | 阿里巴巴集团控股有限公司 | A kind of storage method of business datum, the generation method of record identifier and device |
| CN110928912A (en) * | 2018-09-20 | 2020-03-27 | 北京京东尚科信息技术有限公司 | Method and device for generating unique identifier |
| CN109740120A (en) * | 2018-12-14 | 2019-05-10 | 深圳壹账通智能科技有限公司 | The generation method of unique identification, device, computer equipment under distributed environment |
Non-Patent Citations (1)
| Title |
|---|
| 张刚;黄艳;: "Ecode标识体系存储结构概述及其在工业互联网领域的应用", 中国自动识别技术, no. 06, 15 December 2019 (2019-12-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111447257B (en) | Message conversion method and device | |
| CN110471848B (en) | Method and device for dynamically returning message | |
| CN110837409B (en) | Method and system for executing tasks at fixed time | |
| CN111038906B (en) | Order sorting method and device | |
| CN111460129A (en) | Method and device for generating identification, electronic equipment and storage medium | |
| CN112835904A (en) | Data processing method and data processing device | |
| CN113760948A (en) | Data query method and device | |
| CN110764770A (en) | Method and device for conversion and assignment between objects | |
| CN112988770A (en) | Method and device for updating serial number, electronic equipment and storage medium | |
| CN116303608A (en) | Data processing method and device for application service | |
| CN118260329A (en) | Method, apparatus, device and computer readable medium for processing business data table | |
| CN111241137A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN113704245A (en) | Database main key generation method, sub-table positioning method and device | |
| CN113190517A (en) | Data integration method and device, electronic equipment and computer readable medium | |
| CN116775167A (en) | Service processing method, device, electronic equipment and computer readable medium | |
| CN113641706A (en) | Data query method and device | |
| CN112306984A (en) | Data source routing method and device | |
| CN113742321B (en) | Data updating method and device | |
| CN113742334A (en) | Data processing method and device | |
| CN112988857B (en) | Service data processing method and device | |
| CN110866002B (en) | Method and device for processing sub-table data | |
| CN111984616A (en) | Method, device and system for updating shared file | |
| CN112711572A (en) | Online capacity expansion method and device suitable for sub-warehouse and sub-meter | |
| CN113704242A (en) | Data processing method and device | |
| CN112783914A (en) | Statement optimization method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |