CN114328554A - Data version number updating method and device, electronic equipment and computer storage medium - Google Patents
Data version number updating method and device, electronic equipment and computer storage medium Download PDFInfo
- Publication number
- CN114328554A CN114328554A CN202111646322.XA CN202111646322A CN114328554A CN 114328554 A CN114328554 A CN 114328554A CN 202111646322 A CN202111646322 A CN 202111646322A CN 114328554 A CN114328554 A CN 114328554A
- Authority
- CN
- China
- Prior art keywords
- data
- updated
- field value
- version number
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a method, a device, electronic equipment and a computer storage medium for updating a data version number, wherein the method comprises the steps of determining a storage interval of data to be updated according to a field value of the data to be updated; inquiring the storage interval according to the field value of the data to be updated to obtain historical data of the data to be updated; and executing data updating and version number updating of the data to be updated according to the specified updating type, the data to be updated and the historical data. Therefore, the invention can execute the data updating and version number updating of the data to be updated according to different specified updating types, thereby meeting the use requirements of multiple scenes and having the advantages of low maintenance cost and stable and efficient updating operation.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and an apparatus for updating a data version number, an electronic device, and a computer storage medium.
Background
Most of traditional historical data recording modes adopt a relational database, the version number corresponding to each row of data is recorded by increasing a version number data field, and the recorded version number is single and limited, so that the database is only suitable for a small number of scenes.
In view of this, the version number of the data can be recorded in a multidimensional manner, so as to meet the application requirements of multiple scenes. However, if the conventional data recording method is adopted to record the corresponding version number for each field of data in the database, the table structure is very complex, a large amount of data redundancy is generated, and meanwhile, the service use requirement of joint query cannot be met. In addition, the conventional history data recording method cannot satisfy the update requirement of the list version number (i.e. a uniform version number is recorded for a batch of data).
Therefore, a data updating technique that can meet the requirements of multi-scenario applications and is low in development and maintenance cost is needed.
Disclosure of Invention
In view of the above, one of the technical problems solved by the embodiments of the present invention is to provide a method, an apparatus, an electronic device and a computer storage medium for updating a data version number, which can provide multiple data update types and record the version number of updated data in a multidimensional manner to meet the use requirements of multiple scenes.
According to a first aspect of the present invention, there is provided a data version number updating method, including: determining a storage interval of the data to be updated according to the field value of the data to be updated; inquiring the storage interval according to the field value of the data to be updated to obtain historical data of the data to be updated; and according to the specified updating type and the historical data, executing data updating and version number updating of the data to be updated.
Optionally, the determining, according to the field value in the data to be updated, the storage space of the data to be updated specifically includes: calculating a storage interval value of the data to be updated according to the field value of the key field of the data to be updated; determining the storage interval of the data to be updated according to the storage interval value and a preset database storage rule; the preset database storage rule is used for defining the corresponding relation between each storage interval value and each storage interval.
Optionally, the method further comprises: acquiring each preset updating type supported by the storage engine according to the storage engine of the data to be updated; and selecting one preset updating type to determine the preset updating type of the data to be updated.
Optionally, the querying the storage interval to obtain the historical data of the data to be updated specifically includes: inquiring the storage interval to obtain an inquiry result of the candidate data of the data to be updated, which are stored or not stored in the storage interval; responding to the query result of the candidate data of the data to be updated stored in the storage interval, and querying the number of the version numbers of the candidate data; if the candidate data of one version number is stored in the storage interval, determining the candidate data as the historical data of the data to be updated; if the candidate data with a plurality of version numbers are stored in the storage interval, determining the candidate data with the latest version number as the historical data of the data to be updated according to each version number corresponding to each candidate data.
Optionally, the method further comprises: responding to the query result of the candidate data which does not store the data to be updated in the storage interval, generating an initial version number of the data to be updated, and storing the data to be updated and the initial version number of the data to be updated into the storage interval.
Optionally, the specified update type is a global update; wherein, the executing the data update and version number update of the data to be updated according to the specified update type and the historical data specifically comprises: matching each field value to be updated in the data to be updated with each historical field value in the historical data, and updating each field version number corresponding to each field value to be updated in the data to be updated based on the historical version number of the historical data if each field value to be updated is not completely matched with each historical field value; and storing each field value to be updated and the version number corresponding to each field value to be updated in the data to be updated into the storage interval.
Optionally, the storing, to the storage interval, each to-be-updated field value in the to-be-updated data and a version number corresponding to each to-be-updated field value specifically includes: and generating a storage line adjacent to the historical data in the storage interval, and storing each field value to be updated and the version number corresponding to each field value to be updated in the data to be updated to the storage line.
Optionally, the specified update type is a local update; wherein, the executing the data update and version number update of the data to be updated according to the specified update type and the historical data specifically comprises: matching each field value to be updated in the data to be updated with each history field value in the history data, and if the comparison result shows that each field value to be updated is not completely matched with each history field value, acquiring the unmatched field value to be updated as a target update field value; updating a field version number of the target update field value based on a historical version number of the historical data; and storing the target update field value, the field version number of the target update field value, and the like in the storage interval.
Optionally, the storing the target update field value, the field version number of the target update field value to the storage interval specifically includes: determining the historical field value matched with the target update field value according to the field name of the target update field value and each field name corresponding to each historical field value in the historical data; and generating a storage subunit in a storage unit of the history field value, and storing the target update field value and the field version number of the target update field value to the storage subunit.
Optionally, the data to be updated includes associated data, and the specified update type is list update; wherein, the executing the data update and version number update of the data to be updated according to the specified update type and the historical data specifically comprises: matching each field value to be updated in the data to be updated with each history field value in the history data, and updating each field version number corresponding to each field value to be updated in the data to be updated and the associated version number of the associated data based on the history version number of the history data if each field value to be updated is not completely matched with each history field value; and storing each field value to be updated, a version number corresponding to each field value to be updated, the associated data and the associated version number of the associated data in the data to be updated into the storage interval.
Optionally, the storing, to the storage area, each field value to be updated, a version number corresponding to each field value to be updated, the associated data, and the associated version number of the associated data specifically includes: and generating a storage list in the storage interval so as to store each field value to be updated in the data to be updated, a version number corresponding to each field value to be updated, the associated data and the associated version number of the associated data into the storage list.
Optionally, the specified update type is an overlay update; wherein, the executing the data update and version number update of the data to be updated according to the specified update type and the historical data specifically comprises: and matching each field value to be updated in the data to be updated with each history field value in the history data, if each field value to be updated is not completely matched with each history field value, covering the history data in the storage interval by using the data to be updated, and maintaining the current version number of the history data.
According to a second aspect of the present invention, there is provided a data version number updating apparatus comprising: the historical data acquisition module is used for determining a storage interval of the data to be updated according to the field value of the data to be updated, inquiring the storage interval according to the field value of the data to be updated, and acquiring the historical data of the data to be updated; and the updating module is used for executing data updating and version number updating of the data to be updated according to the specified updating type and the historical data.
According to a third aspect of the present invention, there is provided an electronic device comprising: a memory for storing a computer program; a processor for executing the computer program stored in the memory, and when executed, implementing the method of the first aspect.
According to a fourth aspect of the present invention, there is provided a computer storage medium having stored thereon computer instructions which, when executed by a processor, carry out the method of the first aspect described above.
In summary, the data version number updating method, apparatus, electronic device and computer storage medium provided in the embodiments of the present invention can execute data updating and version number updating of data to be updated according to different specified updating types, so as to meet the use requirements of multiple scenes, and have the advantages of low maintenance cost and stable and efficient updating operation.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art according to the drawings.
Fig. 1 to 6 are schematic flowcharts illustrating a data version number updating method according to first to sixth embodiments of the present invention.
Fig. 7 shows a schematic architecture diagram of a data version number updating apparatus according to a seventh embodiment of the present invention.
Fig. 8 shows an architecture diagram of an electronic device according to an eighth embodiment of the invention.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments of the present invention shall fall within the scope of the protection of the embodiments of the present invention.
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
First embodiment
Fig. 1 shows a flowchart of a data version number updating method according to a first embodiment of the present invention.
It should be noted that, before executing the processing steps of this embodiment, corresponding configuration files may be set in advance for the storage data of different dimensions, and exemplary configuration contents of the configuration files may include:
"tableName": a data table name for identifying each stored data;
"dbType" and "dbSource": "dbType" is used to identify the storage engine selected to store the data (e.g., hbase storage engine, mysql storage engine, etc.), "dbSource" is used to identify the source address of the data store where the data is stored.
In this embodiment, the corresponding data storage source address may be further selected according to the storage engine selected for storing data, for example, when the hbase storage engine is selected for storage, the hbase storage engine may further select to store the storage data in the data storage source address of hbase1 or hbase 2.
"updateLevel": for identifying the preset update types that the current data table can support, such as: global updates (also referred to as column updates), local updates (also referred to as row updates), list updates, overlay updates (also referred to as in-place updates).
"isSupportHi storeupdate": the version number used for identifying whether the version number of the data to be updated is smaller than the current latest version number of the historical data or not is supported. Specifically, when the version number of the data to be updated is smaller than the current latest version number of the historical data, the data to be updated with the version number of 9 is supported to perform updating on the historical data with the version number of 10; on the contrary, when the version number of the data to be updated which is not supported is smaller than the current latest version number of the history data, the updating of the history data with the version number of 10 by using the data to be updated with the version number of 9 is not supported.
"sharedColumns": the method is used for identifying the storage interval for storing the data and can be used for dynamically expanding the size of the data storage space. For example, "shared columns" is used to further identify which storage extent, e.g., C or D, in hbase1 the storage data is stored in when the storage data is stored in the data storage source address of hbase 1.
"sharedRule": the method is used for configuring the size of the preset database and storing the corresponding preset database storage rule according to the structure of the preset database.
As shown in fig. 1, the data version number updating method of this embodiment mainly includes the following steps:
step S102, determining a storage interval of the data to be updated according to the field value of the data to be updated.
Optionally, the data to be updated includes a row of data to be updated.
In this embodiment, the data to be updated includes a plurality of fields, such as "name", "age", "job title", and the like.
In this embodiment, a field in the data to be updated may be designated as a key field, for example, a "name" field in the data to be updated is selected as the key field, other fields, for example, an "index" field, may also be adopted as the key field, and a combination of a plurality of fields, for example, a "name" field and an "age" field, may also be adopted as the key field, which is not limited in the present invention. That is to say, in this embodiment, a single field or a combination of multiple fields that can be used as a unique identifier in multiple fields included in the data to be updated can be used as the key field.
Specifically, the storage interval value of the data to be updated may be calculated first according to the field value of the key field of the data to be updated.
More specifically, the storage interval value of the data to be updated, i.e., the MD5 value, may be first calculated by the MD5 algorithm according to the field value of the key field of the data to be updated. The MD5 algorithm is a hash (hash) algorithm, and for each piece of information, the corresponding MD5 value can be calculated, and the MD5 value must be a 32-bit string composed of a lower case letter and a number, and the MD5 values calculated for the same piece of information are the same and are not different. For example, if the field value of the key field (i.e., "name" field) of the data to be updated is "zhang san", then according to "zhang san", the storage interval value (i.e., MD5 value) of the data to be updated can be calculated as M1 by the MD5 algorithm; if the field value of the key field of the data to be updated is "Liqun", then according to "Liqun", the storage interval value (i.e. MD5 value) of the data to be updated can be calculated to be M2 by the MD5 algorithm.
Then, the storage interval of the data to be updated can be determined according to the calculated storage interval value and the preset database storage rule.
In this embodiment, the preset database storage rule is used to define a corresponding relationship between each storage interval value and each storage interval. The preset setting mode of the storage rule of the database is as follows: according to the configuration content "sharedRule" of the configuration file of the preset database, a preset database structure can be obtained, for example: the preset database structure can be a database with an 8 × 8 structure, wherein 8 × 8 means that 8 databases are established, each database comprises 8 data tables, 64 data tables are counted, the structures of the 64 data tables are completely the same, and data of the same dimension are dispersedly stored in the 64 data tables; according to the configuration content 'shared columns' of the configuration file of the preset database, all the obtained MD5 values are averagely distributed to each data table through an MD5 algorithm, and the data tables of each preset database are guaranteed to be distributed to the MD5 values with the same number, so that the MD5 value interval corresponding to each data table is obtained.
It should be understood that all MD5 values calculated by the MD5 algorithm have a power of 32 of 36, which is infinite, and therefore, it can be guaranteed that MD5 values can be equally distributed to each data table, and that the average distribution referred to here is not an absolute average distribution, but allows an average distribution of preset difference values, for example: if the first data table of the first database of the preset database is allocated to 10000 md5 values, the second data table of the second database of the preset database is allocated to 10005 md5 values, and the preset difference value is 10, it is determined that the first data table of the first database of the preset database and the second data table of the second database of the preset database are allocated to the same md5 values, and so on, and the description is omitted here.
In addition, the preset database structure can be not only a database with an 8 × 8 structure, but also a database with a 16 × 16 structure, where 16 × 16 indicates that 16 databases are established, each database includes 16 data tables, and 256 data tables are totally identical in structure, and data of the same dimension is dispersedly stored in the 256 data tables. The skilled person can also select other preset database structures according to actual needs, and the preset database structures are not limited herein.
Specifically, after the storage interval value (namely, the MD5 value) of "zhang san" is calculated to be M1, a preset database structure is obtained according to the configuration content "shared rule" of the configuration file of the preset database, and an MD5 value interval corresponding to each data table is obtained through an MD5 algorithm according to the configuration content "shared columns" of the configuration file of the preset database; the md5 interval of each data table of the preset database is traversed, and the M1 is inquired in a first data table (such as a C table) of which the storage interval of the preset database is a first database. Similarly, after the storage interval value (namely the MD5 value) of "liquad" is calculated to be M2, obtaining a preset database structure according to the configuration content "shared rule" of the configuration file of the preset database, and obtaining an MD5 value interval corresponding to each data table through an MD5 algorithm according to the configuration content "shared columns" of the configuration file of the preset database; and traversing the md5 interval of each data table of the preset database, and querying that the M2 is in a second data table (such as a D table) of which the storage interval of the preset database is the first database.
And step S104, inquiring a storage interval according to the field value of the data to be updated, and acquiring historical data of the data to be updated.
Specifically, whether history data of "zhangsi" is stored in the C table or not may be queried according to a field value of "zhangsi", or whether history data of "zhangsi" is stored in the D table or not may be queried according to a field value of "zhangsi".
More specifically, it may be queried whether a storage interval value (i.e., md5 value) identical to the storage interval value (i.e., md5 value) exists in the C table according to the field value of "zhang" or, if so, the row of data in the C table is history data of zhang "or it may be queried whether a storage interval value (i.e., md5 value) identical to the storage interval value (i.e., md5 value) exists in the D table according to the storage interval value (i.e., md 3526 value) calculated according to the field value of" zhang ", or, if so, the row of data in the D table is history data of" zhang ".
And step S106, according to the specified updating type, the data to be updated and the historical data, updating the data and version number of the data to be updated.
Optionally, according to a storage engine of the data to be updated, obtaining each preset update type supported by the storage engine, and selecting one preset update type to determine the preset update type as the specified update type of the data to be updated.
In this embodiment, the corresponding storage engine may be selected according to the application scenario of the storage data.
Alternatively, the storage engine may include, but is not limited to: hbase storage engine, mysql storage engine, drive storage engine, Cassandra storage engine.
Optionally, the preset update type may include one of a global update, a local update, a list update, and an overlay update.
In this embodiment, the predetermined update types supported by different storage engines may be different, for example, the hbase storage engine, the Druid storage engine, and the casandra storage engine may support four predetermined update types, i.e., global update, local update, list update, and overlay update, while the mysql storage engine only supports three predetermined update types, i.e., global update, list update, and overlay update.
In this embodiment, different data update operations and version number update operations may be performed on the data to be updated according to different specified update types.
In summary, in the data version number updating method according to the embodiment of the present invention, the corresponding storage interval is determined according to the field value of the data to be updated, so as to obtain the historical data of the data to be updated, and the data updating and version number updating of the data to be updated are performed according to the specified update type. Therefore, the invention can provide various data updating modes, can record the version number of the data in various dimensions so as to meet the use requirements of multiple application scenes, and has the advantages of low development and maintenance cost and high-efficiency and stable data updating operation.
Second embodiment
Fig. 2 is a flowchart illustrating a data version number updating method according to a second embodiment of the present invention. The data version number updating method of this embodiment is a specific implementation of the step S104. As shown in fig. 2, the present embodiment mainly includes the following steps:
step S202, querying the storage interval, determining whether the candidate data to be updated is stored in the storage interval, if the candidate data to be updated is not stored in the storage interval, performing step S204, and if the candidate data to be updated is stored in the storage interval, performing step S206.
Step S204, responding to the query result of the candidate data which is not stored with the data to be updated in the storage interval, generating the initial version number of the data to be updated, and storing the data to be updated and the initial version number of the data to be updated in the storage interval.
Specifically, if candidate data of the data to be updated is not queried in the storage section, for example, if candidate data of "zhang san" (which may also be regarded as history data of the data to be updated) is not queried in the C table (storage section), the data to be updated is directly newly added and stored in the C table, and a version number "0" is specified as an initial version number of the data to be updated.
Step S206, responding to the query result of the candidate data stored with the data to be updated in the storage interval, and querying the number of the version numbers of the candidate data.
Step S208 is to determine whether there are multiple version number candidates in the storage section, if there are multiple version number candidates, step S210 is executed, and if there is only one version number candidate, step S212 is executed.
Step S210, according to each version number corresponding to each candidate data, determining the candidate data with the latest version number as the history data of the data to be updated.
Specifically, when candidate data with a plurality of version numbers exist in the storage area is inquired, the candidate data with the maximum version number is obtained according to each version number corresponding to each candidate data, and the candidate data is used as historical data of the data to be updated.
In step S212, the candidate data is determined as the history data of the data to be updated.
Specifically, when only one candidate data of the version number exists in the storage interval, the candidate data is directly used as the historical data of the data to be updated.
In summary, the data version number updating method of the embodiment queries the candidate data of the data to be updated in the storage interval according to the determined storage interval of the data to be updated, so as to provide a judgment basis for the subsequent data updating operation, thereby improving the execution efficiency of the data updating operation.
Third embodiment
Fig. 3 shows a flowchart of a data version number updating method according to a third embodiment of the present invention, and this embodiment shows a specific implementation of step S106. As shown in fig. 3, the present embodiment mainly includes the following steps:
in step S302, the designated update type is set as a global update (also referred to as a column update).
Step S304, matching each field value to be updated in the data to be updated with each historical field value in the historical data, and updating a field version number corresponding to each field value to be updated in the data to be updated based on the historical version number of the historical data if each field value to be updated and each historical field value in the historical data are not completely matched (i.e., are partially the same and partially different).
For example, assume that the historical data is represented as:
| name (I) | Age (age) | Department of Job's republic of Job | Job level |
| Zhang San | 31 | B | Grade 3 |
The data to be updated is represented as:
| name (I) | Age (age) | Department of Job's republic of Job | Job level |
| Zhang San | 30 | A | Grade 3 |
According to the data to be updated and the historical data, the matching result that the field values to be updated of the age and the department of any job in the data to be updated are not matched with the historical data can be obtained.
In this embodiment, based on the historical version number of the historical data, for example, "9", the version number corresponding to the value to be updated of "31", the value to be updated of "B", and the value to be updated of "3 level" in the data to be updated may be updated to "10".
Step S306, storing each field value to be updated in the data to be updated and the version number corresponding to each field value to be updated in the storage interval.
In this embodiment, a storage row adjacent to the history data may be generated in the storage area, and each field value to be updated in the data to be updated and a version number corresponding to each field value to be updated are stored in the storage row, and each field value to be updated corresponds to one version number.
Fourth embodiment
Fig. 4 shows a flowchart of a data version number updating method according to a fourth embodiment of the present invention, and this embodiment shows another specific implementation of step S106. As shown in fig. 4, the present embodiment mainly includes the following steps:
in step S402, the designated update type is set as a local update (also referred to as a row update).
Step S404, matching each field value to be updated in the data to be updated with each history field value in the history data, and if each field value to be updated is not completely matched with each history field value, acquiring the unmatched field value to be updated as the target update field value.
Specifically, taking the data to be updated and the history data shown in the third embodiment as an example, since the field values to be updated of "age" and "department" in the data to be updated are not matched with the history data, the field values to be updated that are not matched in the data to be updated, that is, the field values to be updated of "31" and the field values to be updated of "B" are obtained as the target update field values.
In step S406, the field version number of the target update field value is updated based on the history version number of the history data.
For example, each field version number corresponding to the field value of "31" and the field value of "B" in the data to be updated may be updated to "10" based on the history version number of "9" of the history data.
Step S408, storing the target update field value and the field version number of the target update field value in the storage interval.
In this embodiment, the history field value matching the target update field value may be determined according to the field name of the target update field value and the field names corresponding to the history field values in the history data, and a storage subunit may be generated in the storage unit of the history field value to store the target update field value and the field version number of the target update field value in the storage subunit.
For example, the history field value matching the target update field value "31" may be determined to be "30" according to the field name "age" of the target update field value "31", and a storage subunit may be generated in the storage unit for storing "30" in the storage interval to store the target field value of "31" and the field version number "10" thereof.
Alternatively, field values of two version numbers of "age", i.e., a history field value "30" and a target update field value "31", may be stored by splitting a storage unit for storing history field values into two columns, respectively.
Fifth embodiment
Fig. 5 is a flowchart illustrating a method for updating a data version number according to a fifth embodiment of the present invention, where this embodiment illustrates another specific implementation of step S106. As shown in fig. 5, the present embodiment mainly includes the following steps:
in this embodiment, the data to be updated may include related data. For example, assuming that "zhang san" has ten subordinate employees, the data of the ten subordinate employees may be used as the associated data of "zhang san".
Step S502, the designated update type is set as list update.
Step S504, matching each field value to be updated in the data to be updated with each historical field value in the historical data, and updating the version number corresponding to each field value to be updated in the data to be updated and the associated version number of the associated data based on the historical version number of the historical data if each field value to be updated is not completely matched with each historical field value.
Specifically, taking the data to be updated and the historical data shown in the third embodiment as an example, since the field values to be updated of the "age" and the "department of occupational department" in the data to be updated are not matched with the historical data, based on the historical version number "9" of the historical data, the version numbers corresponding to the field values to be updated in the data to be updated and the associated version numbers of the associated data are updated to "10".
Step S506, storing each field value to be updated in the data to be updated, the version number corresponding to each field value to be updated, the associated data, and the associated version number of the associated data in the storage interval.
In this embodiment, a storage list may be generated in the storage area to store each field value to be updated in the data to be updated, each field version number corresponding to each field value to be updated, associated data, and associated version number of the associated data in the storage list.
Sixth embodiment
Fig. 6 shows a flowchart of a data version number updating method according to a sixth embodiment of the present invention, and this embodiment shows another specific implementation of step S106. As shown in fig. 6, the present embodiment mainly includes the following steps:
in step S602, the designated update type is set as an overlay update (also referred to as a home update).
Step S604, matching each field value to be updated in the data to be updated with each historical field value in the historical data, if each field value to be updated is not completely matched with each historical field value, covering the historical data in the storage interval by using the data to be updated, and maintaining the current version number of the historical data.
Specifically, taking the data to be updated and the history data shown in the third embodiment as an example, since the field values to be updated of "age" and "department of occupational" in the data to be updated do not match with the history data, the field values to be updated in the data to be updated are used to cover the history field values of the history data in the storage interval, and the current version number of the history data is kept unchanged. It should be appreciated that when an update type is designated as an overlay update, the recording of the version number may be omitted as it is an overlay to the historical data.
It should be noted that, in the third to sixth embodiments, in addition to matching field values in the data to be updated and the history data with the same field name, matching may also be performed on respective fields included in the data to be updated and the history data.
For example, if the history data includes three fields of "age", "department of employment" and "level of employment", and the data to be updated includes four fields of "age", "sex", "department of employment" and "level of employment", a matching result can be obtained that the field value to be updated corresponding to the field of "sex" in the data to be updated does not match the history data.
In summary, by using the third to sixth embodiments of the present invention, different data update and version number update operations can be performed corresponding to different update types, and changes of each dimension of data can be completely recorded, for example, independent version number management can be performed for each field, each group of data, or each batch of data, compared with the conventional data update technology that only one data row can be singly recorded and fields cannot be increased or decreased, the present invention can adopt a multi-dimensional version number recording manner and arbitrarily expand the fields, only one data table is needed to express the meaning of the version numbers of various dimensions, and the update condition of data can be maximally recorded to meet the use requirements of different scenes.
Seventh embodiment
Fig. 7 shows a schematic architecture of a data version number updating apparatus 700 according to a seventh embodiment of the present invention. As shown in fig. 7, the data version number updating apparatus 700 of the present embodiment mainly includes a history data obtaining module 702 and an updating module 704.
The historical data obtaining module 702 is configured to determine a storage interval of the data to be updated according to the field value of the data to be updated, query the storage interval according to the field value of the data to be updated, and obtain the historical data of the data to be updated.
Optionally, the historical data obtaining module 702 is further configured to calculate a storage interval value of the data to be updated according to the field value of the key field of the data to be updated; determining the storage interval of the data to be updated according to the storage interval value and a preset database storage rule; the preset database storage rule is used for defining the corresponding relation between each storage interval value and each storage interval.
Optionally, the historical data obtaining module 702 is further configured to query the storage interval, and obtain a query result of candidate data in which the data to be updated is stored or not stored in the storage interval; responding to the query result of the candidate data of the data to be updated stored in the storage interval, and querying the number of the version numbers of the candidate data; if the candidate data of one version number is stored in the storage interval, determining the candidate data as the historical data of the data to be updated; if the candidate data with a plurality of version numbers are stored in the storage interval, determining the candidate data with the latest version number as the historical data of the data to be updated according to each version number corresponding to each candidate data.
The updating module 704 is configured to perform data updating and version number updating of the data to be updated according to a specified updating type, the data to be updated, and the history data.
Optionally, the updating module 704 is further configured to obtain, according to the storage engine of the data to be updated, each preset update type supported by the storage engine; and selecting one preset updating type to determine the preset updating type of the data to be updated.
Optionally, the storage engine comprises one of a hbase storage engine, a mysql storage engine, a Druid storage engine, a Cassandra storage engine.
Optionally, the updating module 704 is further configured to respond to a query result of the candidate data, in which the data to be updated is not stored, in the storage interval of the historical data obtaining module 702, generate an initial version number of the data to be updated, and store the data to be updated and the initial version number of the data to be updated into the storage interval.
Optionally, the specified update type is an overall update, and the update module 704 is further configured to match each field value to be updated in the data to be updated with each history field value in the history data, and if each field value to be updated is not completely matched with each history field value, update each field version number corresponding to each field value to be updated in the data to be updated based on a history version number of the history data; and storing each field value to be updated and the version number corresponding to each field value to be updated in the data to be updated into the storage interval.
Optionally, the updating module 704 is further configured to generate a storage line adjacent to the historical data in the storage interval, and store each field value to be updated in the data to be updated and a version number corresponding to each field value to be updated in the storage line.
Optionally, the specified update type is a local update, the update module 704 is further configured to match each field value to be updated in the data to be updated with each history field value in the history data, and if the comparison result indicates that each field value to be updated is not completely matched with each history field value, obtain the unmatched field value to be updated as a target update field value; updating a field version number of the target update field value based on a historical version number of the historical data; and storing the target update field value, the field version number of the target update field value, and the like in the storage interval.
Optionally, the updating module 704 is further configured to determine, according to a field name of the target update field value and each field name corresponding to each historical field value in the historical data, the historical field value that matches the target update field value; and generating a storage subunit in a storage unit of the history field value, and storing the target update field value and the field version number of the target update field value to the storage subunit.
Optionally, the data to be updated includes associated data, the updating module 704 is further configured to match each field value to be updated in the data to be updated with each history field value in the history data, and if each field value to be updated is not completely matched with each history field value, update, based on a history version number of the history data, a version number corresponding to each field value to be updated in the data to be updated and an associated version number of the associated data; and storing each field value to be updated in the data to be updated, each field version number corresponding to each field value to be updated, the associated data and the associated version number of the associated data in the storage interval.
Optionally, the updating module 704 is further configured to generate a storage list in the storage interval, so as to store each field value to be updated in the data to be updated, each field version number corresponding to each field value to be updated, the associated data, and the associated version number of the associated data in the storage list.
Optionally, the update-type-designated-to-be-overwritten update module 704 is further configured to match each field value to be updated in the data to be updated with each history field value in the history data, and if each field value to be updated is not completely matched with each history field value, overwrite the history data in the storage interval with the data to be updated, and maintain the current version number of the history data.
In addition, the data version number updating apparatus 700 according to the embodiment of the present invention may also be used to implement other steps in the foregoing data version number updating method embodiments, and has the beneficial effects of the corresponding method step embodiments, which are not described herein again.
Eighth embodiment
An eighth embodiment of the present invention provides an electronic device, which mainly includes a memory and a processor, wherein the memory is used for storing a computer program, the processor is used for executing the computer program stored in the memory, and when the computer program is executed, the method described in any one of the first to sixth embodiments is implemented.
Fig. 8 is a schematic block diagram illustrating an electronic device 800 according to an eighth embodiment of the present invention, and as shown in fig. 8, the electronic device 800 of this embodiment may include a processor (processor) 802, a communication interface (communication interface)804, and a memory (memory) 806.
The processor 802, communication interface 804, and memory 806 communicate with one another via a communication bus 808.
A communication interface 804 for communicating with other electronic devices, such as a terminal device or a server.
The processor 802 is configured to execute the computer program 810, and may specifically perform the relevant steps in the above-described embodiments of the methods.
In particular, the computer program 810 may comprise program code comprising computer operating instructions.
The processor 802 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement embodiments of the present invention. The electronic device comprises one or more processors, which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
A memory 806 for storing a computer program 810. The memory 806 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
Ninth embodiment
A ninth embodiment of the present invention provides a computer storage medium having computer instructions stored thereon, which when executed by a processor, can implement the method of any one of the first to sixth embodiments.
In summary, the data version number updating method, apparatus, electronic device and computer storage medium provided in the embodiments of the present invention can perform data updating and version number updating in different updating manners according to different specified updating types, so that not only can changes of each dimension of data be completely recorded to simultaneously meet the usage requirements of multiple scenes, but also fields can be increased or decreased at will, and the method, apparatus, electronic device and computer storage medium have the advantages of low development and maintenance cost, and stable and efficient updating operation
It should be noted that, according to the implementation requirement, each component/step described in the embodiment of the present invention may be divided into more components/steps, and two or more components/steps or partial operations of the components/steps may also be combined into a new component/step to achieve the purpose of the embodiment of the present invention.
The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware such as an ASIC or FPGA. It is understood that the computer, processor, microprocessor controller or programmable hardware includes memory components (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the data version number update methods described herein. Further, when a general-purpose computer accesses code for implementing the data version number updating method shown herein, execution of the code converts the general-purpose computer into a special-purpose computer for performing the data version number updating method shown herein.
Those of ordinary skill in the art will appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.
The above embodiments are only for illustrating the embodiments of the present invention and not for limiting the embodiments of the present invention, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention, so that all equivalent technical solutions also belong to the scope of the embodiments of the present invention, and the scope of patent protection of the embodiments of the present invention should be defined by the claims.
Claims (15)
1. A method for determining a data update version number is characterized by comprising the following steps:
determining a storage interval of the data to be updated according to the field value of the data to be updated;
inquiring the storage interval according to the field value of the data to be updated to obtain historical data of the data to be updated; and
and according to the specified updating type, the data to be updated and the historical data, executing data updating and version number updating of the data to be updated.
2. The method for updating a data version number according to claim 1, wherein the determining a storage space of the data to be updated according to the field value of the data to be updated specifically comprises:
calculating a storage interval value of the data to be updated according to the field value of the key field of the data to be updated;
determining the storage interval of the data to be updated according to the storage interval value and a preset database storage rule;
the preset database storage rule is used for defining the corresponding relation between each storage interval value and each storage interval.
3. A method of updating a data version number according to any of claims 1-2, further comprising:
acquiring each preset updating type supported by the storage engine according to the storage engine of the data to be updated;
and selecting one preset updating type to determine the preset updating type of the data to be updated.
4. The method for updating a data version number according to any one of claims 1 to 3, wherein the querying the storage interval according to the field value of the data to be updated to obtain the historical data of the data to be updated specifically comprises:
inquiring the storage interval to obtain an inquiry result of the candidate data of the data to be updated, which are stored or not stored in the storage interval;
responding to the query result of the candidate data of the data to be updated stored in the storage interval, and querying the number of the version numbers of the candidate data;
if the candidate data of one version number is stored in the storage interval, determining the candidate data as the historical data of the data to be updated;
if the candidate data with a plurality of version numbers are stored in the storage interval, determining the candidate data with the latest version number as the historical data of the data to be updated according to each version number corresponding to each candidate data.
5. The data version number updating method according to claim 4, further comprising:
responding to the query result of the candidate data which does not store the data to be updated in the storage interval, generating an initial version number of the data to be updated, and storing the data to be updated and the initial version number of the data to be updated into the storage interval.
6. The data version number updating method according to any one of claims 1 to 4, wherein the specified update type is a whole update; wherein the content of the first and second substances,
the executing the data update and version number update of the data to be updated according to the specified update type and the historical data specifically comprises:
matching each field value to be updated in the data to be updated with each historical field value in the historical data, and updating a version number corresponding to each field value to be updated in the data to be updated based on a historical version number of the historical data if each field value to be updated is not completely matched with each historical field value; and
and storing each field value to be updated and the version number corresponding to each field value to be updated in the data to be updated into the storage interval.
7. The method for updating a data version number according to claim 6, wherein the storing, to the storage interval, each field value to be updated and a version number corresponding to each field value to be updated specifically comprises:
and generating a storage line adjacent to the historical data in the storage interval, and storing each field value to be updated and the version number corresponding to each field value to be updated in the data to be updated to the storage line.
8. The data version number updating method according to any one of claims 1 to 4, wherein the specified update type is a local update; wherein the content of the first and second substances,
the executing the data update and version number update of the data to be updated according to the specified update type and the historical data specifically comprises:
matching each field value to be updated in the data to be updated with each history field value in the history data, and if the comparison result shows that each field value to be updated is not completely matched with each history field value, acquiring the unmatched field value to be updated as a target update field value;
updating a field version number of the target update field value based on a historical version number of the historical data; and
and storing the target update field value and the field version number of the target update field value into the storage interval.
9. The method of claim 8, wherein storing the target update field value, the field version number of the target update field value, and the storage interval specifically comprises:
determining the historical field value matched with the target update field value according to the field name of the target update field value and each field name corresponding to each historical field value in the historical data;
and generating a storage subunit in a storage unit of the history field value, and storing the target update field value and the field version number of the target update field value to the storage subunit.
10. The data version number updating method according to any one of claims 1 to 4, wherein the data to be updated includes associated data, and the specified update type is a list update; wherein the content of the first and second substances,
the executing the data update and version number update of the data to be updated according to the specified update type, the data to be updated and the historical data specifically comprises:
matching each field value to be updated in the data to be updated with each history field value in the history data, and updating each field version number corresponding to each field value to be updated in the data to be updated and the associated version number of the associated data based on the history version number of the history data if each field value to be updated is not completely matched with each history field value; and
and storing each field value to be updated, a version number corresponding to each field value to be updated, the associated data and the associated version number of the associated data in the data to be updated into the storage interval.
11. The method for updating a data version number according to claim 10, wherein the storing, in the storage area, each field value to be updated, a version number corresponding to each field value to be updated, the associated data, and the associated version number of the associated data specifically includes:
and generating a storage list in the storage interval so as to store each field value to be updated in the data to be updated, a version number corresponding to each field value to be updated, the associated data and the associated version number of the associated data into the storage list.
12. The data version number updating method according to any one of claims 1 to 4, wherein the specified update type is an overlay update; wherein the content of the first and second substances,
the executing the data update and version number update of the data to be updated according to the specified update type and the historical data specifically comprises:
and matching each field value to be updated in the data to be updated with each history field value in the history data, if each field value to be updated is not completely matched with each history field value, covering the history data in the storage interval by using the data to be updated, and maintaining the current version number of the history data.
13. An apparatus for updating a data version number, comprising:
the historical data acquisition module is used for determining a storage interval of the data to be updated according to a field value of the data to be updated, inquiring the storage interval according to the field value of the data to be updated, and acquiring historical data of the data to be updated;
and the updating module is used for executing data updating and version number updating of the data to be updated according to the specified updating type, the data to be updated and the historical data.
14. An electronic device, comprising:
a memory for storing a computer program;
a processor for executing a computer program stored in the memory, and which, when executed, implements the method of any of the preceding claims 1 to 12.
15. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of the preceding claims 1 to 12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111646322.XA CN114328554A (en) | 2021-12-29 | 2021-12-29 | Data version number updating method and device, electronic equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111646322.XA CN114328554A (en) | 2021-12-29 | 2021-12-29 | Data version number updating method and device, electronic equipment and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114328554A true CN114328554A (en) | 2022-04-12 |
Family
ID=81017072
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111646322.XA Pending CN114328554A (en) | 2021-12-29 | 2021-12-29 | Data version number updating method and device, electronic equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114328554A (en) |
-
2021
- 2021-12-29 CN CN202111646322.XA patent/CN114328554A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9507875B2 (en) | Symbolic hyper-graph database | |
| US11423085B2 (en) | Graph database super vertex partitioning | |
| EP3236365A1 (en) | Data query method and device | |
| CN108388604B (en) | User authority data management apparatus, method and computer readable storage medium | |
| CN111241350B (en) | Graph data query method, device, computer equipment and storage medium | |
| CN114077680B (en) | Graph data storage method, system and device | |
| CN112256698B (en) | Table relation automatic association method based on multi-hash function | |
| CN113190720B (en) | Graph compression-based graph database construction method and device and related components | |
| AU2014353667A1 (en) | A method of generating a reference index data structure and method for finding a position of a data pattern in a reference data structure | |
| CN104268298A (en) | Method for creating database index and inquiring data | |
| CN105005567B (en) | Interest point query method and system | |
| CN113268439A (en) | Memory address searching method and device, electronic equipment and storage medium | |
| CN107562762B (en) | Data index construction method and device | |
| US10698955B1 (en) | Weighted abstract path graph database partitioning | |
| CN114328554A (en) | Data version number updating method and device, electronic equipment and computer storage medium | |
| WO2016022019A1 (en) | Method for data input into a database | |
| CN114048219A (en) | Graph database updating method and device | |
| CN107092604B (en) | File processing method and device | |
| CN112639761B (en) | Method and device for establishing index for data | |
| CN114860806A (en) | Data query method and device of block chain, computer equipment and storage medium | |
| CN114116702A (en) | Key value database-based quick query method for adjacent points | |
| CN111858609A (en) | Fuzzy query method and device for block chain | |
| CN110825747A (en) | Information access method, device and medium | |
| CN107085571B (en) | Method and device for executing check rule | |
| Abawajy et al. | A framework for scalable distributed provenance storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |