CN112465630B - Index data processing method, device, equipment and storage medium - Google Patents
Index data processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112465630B CN112465630B CN202011444448.4A CN202011444448A CN112465630B CN 112465630 B CN112465630 B CN 112465630B CN 202011444448 A CN202011444448 A CN 202011444448A CN 112465630 B CN112465630 B CN 112465630B
- Authority
- CN
- China
- Prior art keywords
- index set
- index
- data
- data table
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title description 6
- 238000000034 method Methods 0.000 claims abstract description 36
- 238000007405 data analysis Methods 0.000 claims 1
- 230000008859 change Effects 0.000 abstract description 41
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000012423 maintenance Methods 0.000 description 5
- 239000004973 liquid crystal related substance Substances 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/03—Credit; Loans; Processing thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Marketing (AREA)
- Entrepreneurship & Innovation (AREA)
- General Business, Economics & Management (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Educational Administration (AREA)
- Game Theory and Decision Science (AREA)
- Technology Law (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the field of data processing, and provides a method, a device, equipment and a storage medium for processing index data. The method comprises the following steps: judging whether a new index set exists in the database, and storing relevant information corresponding to the new index set to a first data table and a second data table when the new index set exists; judging whether the description information of the index set in the database is changed or not when the description information does not exist, judging whether index metadata of the index set is changed or not, and storing related information corresponding to the changed index set into a first data table and a third data table when any change exists; judging whether the database has a deleted index set or not, if so, storing the related information of the deleted index set into a fourth data table; when any one data table of the second data table, the third data table and the fourth data table is not empty, generating latest snapshot data and storing the latest snapshot data into the index set snapshot table. The invention can reduce the overhead of the system and realize the information synchronization among different systems.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a method, an apparatus, a device, and a storage medium for processing index data.
Background
In the personal credit field, the assessment of risk level of a user needs to rely on a plurality of characteristic index data, the information describing the index data, namely metadata information, can be changed according to the adjustment of business, and a plurality of systems need to acquire characteristic index metadata to acquire specific meaning of the data. If the characteristic index metadata information is manually maintained, hysteresis occurs and human errors can be introduced, and if the information is maintained by each system using the characteristic index metadata, the maintenance cost of the system and the information synchronization difficulty between the systems can be greatly increased.
Disclosure of Invention
In view of the above, the present invention provides a method, apparatus, device and storage medium for processing index data, which aims to solve the technical problems of high system overhead and difficult information synchronization when managing data in the prior art.
In order to achieve the above object, the present invention provides a method for processing index data, the method comprising:
judging whether a database has a new index set, and storing relevant information corresponding to the new index set into a first data table and a second data table of a preset storage path when judging that the database is in the new index set;
When judging that the database does not have a newly added index set, judging whether the description information of the index set in the database is changed or not, judging whether index metadata of the index set is changed or not, and when judging that the description information of the index set or the index metadata information of the index set is changed, storing related information corresponding to the changed index set into a first data table and a third data table of the preset storage path;
judging whether the database has a deleted index set or not, if so, storing the related information of the deleted index set into a fourth data table of the preset storage path;
and when any one data table among the second data table, the third data table and the fourth data table is not empty, generating latest snapshot data and storing the latest snapshot data into a preset index set snapshot table.
Preferably, after determining that the database has a new index set, the method further includes:
and storing the index metadata of the newly-increased index set into an index detail table of the database.
Preferably, after determining that there is a change in the description information of the index set or the index metadata information of the index set, the method further includes:
And analyzing and obtaining the latest data after the index set is changed according to a predefined data structure, and storing the latest data into an index detail table of the database.
Preferably, the determining whether the description information of the scanned index set has a change includes:
acquiring primary key information corresponding to the index set from an index set list corresponding to the index set, and acquiring description information of snapshot data corresponding to the index set from an index detail table of a database based on the primary key information;
judging whether the description information in the snapshot data is the same as the description information in the index set annotation information, and if not, judging that the scanned description information of the index set has change.
Preferably, the determining whether the description information of the index set in the database has a change, and determining whether the index metadata of the index set has a change includes:
and when the description information of the index set is judged and the index metadata of the index set are unchanged, storing the related information corresponding to the index set into a first data table of the preset storage path, wherein the related information comprises the primary key information of the index set storage record, the English name and the Chinese name of the index set.
In order to achieve the above object, the present invention also provides a processing device for index data, the processing device for index data comprising:
a first judging module: the method comprises the steps of judging whether a database has a new index set or not, and storing relevant information corresponding to the new index set into a first data table and a second data table of a preset storage path when the database is judged to be in the new index set;
and a second judging module: when judging that the description information of the index set in the database is changed, and whether index metadata of the index set is changed, and when judging that the description information of the index set or the index metadata information of the index set is changed, storing related information corresponding to the changed index set into a first data table and a third data table of the preset storage path;
and a third judging module: the fourth data table is used for judging whether the database has the deleted index set or not, if so, the related information of the deleted index set is stored in the fourth data table to the preset storage path;
the generation module is used for: and generating latest snapshot data and storing the latest snapshot data into a preset index set snapshot table when any one of the second data table, the third data table and the fourth data table is not empty.
To achieve the above object, the present invention also provides an electronic device including:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a program executable by the at least one processor, the program being executable by the at least one processor to enable the at least one processor to perform the steps of:
judging whether a database has a new index set, and storing relevant information corresponding to the new index set into a first data table and a second data table of a preset storage path when judging that the database is in the new index set;
when judging that the database does not have a newly added index set, judging whether the description information of the index set in the database is changed or not, judging whether index metadata of the index set is changed or not, and when judging that the description information of the index set or the index metadata information of the index set is changed, storing related information corresponding to the changed index set into a first data table and a third data table of the preset storage path;
judging whether the database has a deleted index set or not, if so, storing the related information of the deleted index set into a fourth data table of the preset storage path;
And when any one data table among the second data table, the third data table and the fourth data table is not empty, generating latest snapshot data and storing the latest snapshot data into a preset index set snapshot table.
Preferably, the determining whether the description information of the scanned index set has a change includes:
acquiring primary key information corresponding to the index set from an index set list corresponding to the index set, and acquiring description information of snapshot data corresponding to the index set from an index detail table of a database based on the primary key information;
judging whether the description information in the snapshot data is the same as the description information in the index set annotation information, and if not, judging that the scanned description information of the index set has change.
Preferably, the determining whether the description information of the index set in the database has a change, and determining whether the index metadata of the index set has a change includes:
and when the description information of the index set is judged and the index metadata of the index set are unchanged, storing the related information corresponding to the index set into a first data table of the preset storage path, wherein the related information comprises the primary key information of the index set storage record, the English name and the Chinese name of the index set.
In order to achieve the above object, the present invention also provides a computer-readable storage medium storing a processing program of index data, which when executed by a processor, implements any step of the processing method of index data as described above.
The processing method, the device, the equipment and the storage medium for the index data automatically scan the file marked by the specific annotation to generate the corresponding index metadata for storage, so that the index metadata are always consistent with the latest index data of the system, the time difference problem of multi-system maintenance is avoided, the maintenance cost of the system can be reduced, the synchronization of information among different systems is realized, and the developer only needs to modify the changed index metadata, thereby greatly reducing the workload.
Drawings
FIG. 1 is a flow chart illustrating a method for processing index data according to a preferred embodiment of the present invention;
FIG. 2 is a block diagram of a processing device for index data according to a preferred embodiment of the present invention;
FIG. 3 is a schematic diagram of an electronic device according to a preferred embodiment of the present invention;
the achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention provides a method for processing index data. Referring to fig. 1, a method flow chart of an embodiment of a method for processing index data according to the present invention is shown. The method may be performed by an electronic device, which may be implemented in software and/or hardware. The index data processing method comprises the following steps:
step S10: judging whether a database has a new index set, and storing relevant information corresponding to the new index set into a first data table and a second data table of a preset storage path when the database is judged to be in the new index set.
In this embodiment, each index set corresponds to a JAVA class, where the index set may include a personal information feature index set, a social security feature index set, an aggregate feature index set, and so on, all defined index sets may be stored under a certain data packet and managed uniformly, and when a program is started, a plurality of annotated index sets in the index set data packet are scanned, and a first index set closest to the current time in the index set snapshot table is obtained, where determining whether a new index set exists in the database may determine whether the first index set exists in the database, or determining whether an english name in the scanned index set solution information exists in an index set list corresponding to the first index set, if not, it is indicated that the scanned index set is a new annotated index set with respect to the last snapshot data, and then relevant information in the new index set and annotation information of metadata of JAVA class corresponding to the new index set are parsed, and metadata information of the new index set is extracted.
Storing the related information corresponding to the newly added index set into a first data table and a second data table of a preset storage path, wherein the related information corresponding to the newly added index set comprises main key information of a storage record, an English name and a Chinese name of the newly added index set, the preset storage path can be a memory, the first data table can be an index set list, and the second data table can be an newly added index set list.
In one embodiment, after determining that the database has a new set of metrics, the method further comprises: and storing the index metadata of the newly-increased index set into an index detail table of the database.
And storing the newly added index set and index metadata of the newly added index set into an index detail table of the database, wherein the database can return the primary key information of the stored record.
Step S20: when judging that the database does not have the newly added index set, judging whether the description information of the index set in the database is changed or not, judging whether the index metadata of the index set is changed or not, and when judging that the description information of the index set or the index metadata information of the index set is changed, storing the related information corresponding to the changed index set into a first data table and a third data table of the preset storage path.
In this embodiment, when it is determined that there is no newly added index set in the database, that is, when an english name in the scanned index set solution information exists in an index set list corresponding to a first index set, it is determined whether there is a change in description information of the index set in the database, if there is a change, corresponding change information is generated, annotation information of metadata of JAVA class corresponding to the index set is analyzed, then it is determined whether there is a change in index metadata of the index set, when either the description information of the index set or the index metadata of the index set is changed, and related information of the index set after the change is stored in a first data table and a third data table of a preset storage path, where the related information includes primary key information, an english name and a chinese name of a storage record, and the first data table may refer to the index set list in a memory, and the third data table may be a change index set list in the memory.
Further, after determining that there is a change in the description information of the index set or the index metadata information of the index set, the method further includes:
and analyzing and obtaining the latest data after the index set is changed according to a predefined data structure, and storing the latest data into an index detail table of the database.
According to the defined data structure, analyzing the latest data, storing the latest data in an index detail table in a database, and after the latest data is stored in the database, automatically generating a main key ID of the data by the database and returning the main key ID.
In one embodiment, the determining whether there is a change in the description information of the scanned index set includes: acquiring primary key information corresponding to the index set from an index set list corresponding to the index set, and acquiring description information of snapshot data corresponding to the index set from an index detail table of a database based on the primary key information;
judging whether the description information in the snapshot data is the same as the description information in the index set annotation information, and if not, judging that the scanned description information of the index set has change.
Acquiring setId information i1 corresponding to English names in an index set list of the index set, storing a primary key of an index detail table by the setId, acquiring description information corresponding to the index set from the index detail table in a database according to the i1, marking the description information as A1 (A1 is snapshot data), comparing whether the description information in annotation information is identical to the description information in A1, and if not, changing. The description information refers to a cname/table/desc/category, compares whether the value of the corresponding field is changed, and queries with i1 as a primary key.
In one embodiment, when the description information of the index set is determined and the index metadata of the index set is unchanged, storing related information corresponding to the index set to a first data table of a preset storage path, wherein the related information includes primary key information of a storage record of the index set, an english name and a chinese name of the index set.
When the description information of the index set and the index metadata of the index set are unchanged, a new data change record is not generated, and the related information corresponding to the index set is stored in a first data table of a preset storage path. The related information corresponding to the index set comprises the main key information of the storage record, the English name and the Chinese name of the index set. The first data table may refer to a list of index sets in memory.
Step S30: judging whether the database has the deleted index set or not, if so, storing the related information of the deleted index set into the fourth data table of the preset storage path.
In this embodiment, it is determined whether a deleted index set may form a set B1 from all the index sets currently annotated in the data packet, an index set list corresponding to a first index set closest to the current time in the index set snapshot table is obtained, an index set list corresponding to the first index set is formed into a set B2, whether an index set not existing in B1 is determined in B2, if yes, it is indicated that, with respect to the index set snapshot table, the deleted index set exists in the database, relevant information of the deleted index set is stored in a fourth data table of a preset storage path, the preset storage path may refer to a memory, and the fourth data table may refer to the deleted index set list in this embodiment.
Step S40: and when any one data table among the second data table, the third data table and the fourth data table is not empty, generating latest snapshot data and storing the latest snapshot data into a preset index set snapshot table.
In this embodiment, if any one of the second data table (newly added index set list), the third data table (changed index set list) and the fourth data table (deleted index set list) in the memory is not empty, which indicates that the current start has occurred with the change of the rule index metadata or the metadata set, a new record (latest snapshot data) may be generated and stored in the index set snapshot table, and if all of the second data table (newly added index set list), the third data table (changed index set list) and the fourth data table (deleted index set list) are empty, which indicates that the current start has no change of any metadata and no new data is generated.
The database records index set snapshot data and detailed index set data, and all detailed data and change data are recorded. The developer only needs to modify index metadata with changes, so that the workload is greatly reduced. When the system is started, the file marked by the specific annotation is automatically scanned to generate corresponding index metadata for storage, so that the index metadata is always kept once with the latest index data of the system, and the problem of time difference caused by multi-system maintenance is avoided.
In one embodiment, a plurality of interfaces can be provided for other systems to acquire metadata related information, so that other systems can acquire accurate and detailed index metadata information, and the data can be used everywhere only by one definition. The interface comprises:
current index set snapshot information interface: the interface returns snapshot information of the current system index set. The system acquires the latest piece of data D1 in the index set snapshot table of the database table, queries the data in the detail table of the index set according to setId information in the index set list corresponding to the data D1 to acquire detailed index set information C1-CN, and assembles the data of the D1 and the C1-CN and returns the assembled data. The calling system acquires the current latest index set list information and detailed index data in each index set.
Index set snapshot information history interface: the interface is required to be transferred into the starting time T1 and T2, after the time format is checked, the interface queries the information of the time in the time period in the index set snapshot table of the database table, and returns the data to the calling interface after the data is extracted and formatted.
Index set snapshot information interface: the calling interface needs to be transmitted with the characteristic index set snapshot ID information ID0, inquires the data D0 with the ID of ID0 in the index set snapshot table of the database table, and if the data exists, inquires the data in the index set detail table according to setId information in the index set list of the data D0 to acquire detailed index set information C1-CN, and the data of D0 and C1-CN are returned after being assembled. The calling system will acquire the corresponding index set list information and detailed index data in each index set
Index set snapshot information comparison interface: the calling interface needs to transmit ID information ID1 and ID2 of the characteristic index set snapshot, the interface inquires data D1 and D2 with ID1 and ID2 in the index set snapshot table of the database table, if the data D1 and D2 exist, the data of the two index set lists are compared according to the setId information in the index set list corresponding to the data D1 and D2 to calculate new added, modified and deleted data of the index set list, and then the change of the description information of each index set and the new added, modified and deleted data of the internal index metadata are calculated, and the data is formatted and returned to the calling interface.
Referring to fig. 2, a functional block diagram of an index data processing apparatus 100 according to the present invention is shown.
The index data processing apparatus 100 according to the present invention may be incorporated into an electronic device. The processing device 100 of the index data may include a first judging module 110, a second judging module 120, a third judging module 130 and a generating module 140 according to the implemented functions. The module of the invention, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the electronic device, capable of being executed by the processor of the electronic device and of performing a fixed function.
In the present embodiment, the functions concerning the respective modules/units are as follows:
the first determining module 110 is configured to determine whether a new index set exists in the database, and store relevant information corresponding to the new index set to a first data table and a second data table of a preset storage path when the database is determined to be in the new index set.
In this embodiment, each index set corresponds to a JAVA class, where the index set may include a personal information feature index set, a social security feature index set, an aggregate feature index set, and so on, all defined index sets may be stored under a certain data packet and managed uniformly, and when a program is started, a plurality of annotated index sets in the index set data packet are scanned, and a first index set closest to the current time in the index set snapshot table is obtained, where determining whether a new index set exists in the database may determine whether the first index set exists in the database, or determining whether an english name in the scanned index set solution information exists in an index set list corresponding to the first index set, if not, it is indicated that the scanned index set is a new annotated index set with respect to the last snapshot data, and then relevant information in the new index set and annotation information of metadata of JAVA class corresponding to the new index set are parsed, and metadata information of the new index set is extracted.
Storing the related information corresponding to the newly added index set into a first data table and a second data table of a preset storage path, wherein the related information corresponding to the newly added index set comprises main key information of a storage record, an English name and a Chinese name of the newly added index set, the preset storage path can be a memory, the first data table can be an index set list, and the second data table can be an newly added index set list.
In one embodiment, after determining that the database has a new index set, the first determining module is further configured to store the new index set and index metadata of the new index set into an index detail table of the database.
And storing the newly added index set and index metadata of the newly added index set into an index detail table of the database, wherein the database can return the primary key information of the stored record.
And the second judging module 120 is configured to, when judging that the database does not have a new index set, judge whether the description information of the index set in the database has a change, and judge whether the index metadata of the index set has a change, and when judging that the description information of the index set or the index metadata information of the index set has a change, store the related information corresponding to the changed index set into the first data table and the third data table of the preset storage path.
In this embodiment, when it is determined that there is no newly added index set in the database, that is, when an english name in the scanned index set solution information exists in an index set list corresponding to a first index set, it is determined whether there is a change in description information of the index set in the database, if there is a change, corresponding change information is generated, annotation information of metadata of JAVA class corresponding to the index set is analyzed, then it is determined whether there is a change in index metadata of the index set, when either the description information of the index set or the index metadata of the index set is changed, and related information of the index set after the change is stored in a first data table and a third data table of a preset storage path, where the related information includes primary key information, an english name and a chinese name of a storage record, and the first data table may refer to the index set list in a memory, and the third data table may be a change index set list in the memory.
Further, after determining that the description information of the index set or the index metadata information of the index set has a change, the second determining module is further configured to: and analyzing and obtaining the latest data after the index set is changed according to a predefined data structure, and storing the latest data into an index detail table of the database.
According to the defined data structure, analyzing the latest data, storing the latest data in an index detail table in a database, and after the latest data is stored in the database, automatically generating a main key ID of the data by the database and returning the main key ID.
In one embodiment, the determining whether there is a change in the description information of the scanned index set includes: acquiring primary key information corresponding to the index set from an index set list corresponding to the index set, and acquiring description information of snapshot data corresponding to the index set from an index detail table of a database based on the primary key information;
judging whether the description information in the snapshot data is the same as the description information in the index set annotation information, and if not, judging that the scanned description information of the index set has change.
Acquiring setId information i1 corresponding to English names in an index set list of the index set, storing a primary key of an index detail table by the setId, acquiring description information corresponding to the index set from the index detail table in a database according to the i1, marking the description information as A1 (A1 is snapshot data), comparing whether the description information in annotation information is identical to the description information in A1, and if not, changing. The description information refers to a cname/table/desc/category, compares whether the value of the corresponding field is changed, and queries with i1 as a primary key.
In one embodiment, when the description information of the index set is determined and the index metadata of the index set is unchanged, storing related information corresponding to the index set to a first data table of a preset storage path, wherein the related information includes primary key information of a storage record of the index set, an english name and a chinese name of the index set.
When the description information of the index set and the index metadata of the index set are unchanged, a new data change record is not generated, and the related information corresponding to the index set is stored in a first data table of a preset storage path. The related information corresponding to the index set comprises the main key information of the storage record, the English name and the Chinese name of the index set. The first data table may refer to a list of index sets in memory.
And a third judging module 130, configured to judge whether the database has a deleted index set, if yes, store related information of the deleted index set in the fourth data table of the preset storage path.
In this embodiment, it is determined whether a deleted index set may form a set B1 from all the index sets currently annotated in the data packet, an index set list corresponding to a first index set closest to the current time in the index set snapshot table is obtained, an index set list corresponding to the first index set is formed into a set B2, whether an index set not existing in B1 is determined in B2, if yes, it is indicated that, with respect to the index set snapshot table, the deleted index set exists in the database, relevant information of the deleted index set is stored in a fourth data table of a preset storage path, the preset storage path may refer to a memory, and the fourth data table may refer to the deleted index set list in this embodiment.
And the generating module 140 is configured to generate the latest snapshot data and store the latest snapshot data in a preset index set snapshot table when any one of the second data table, the third data table and the fourth data table is not empty.
In this embodiment, if any one of the second data table (newly added index set list), the third data table (changed index set list) and the fourth data table (deleted index set list) in the memory is not empty, which indicates that the change of the rule index metadata or the metadata set occurs in the current start, a new record (latest snapshot data) may be generated and stored in the index set snapshot table, and if all of the second data table (newly added index set list), the third data table (changed index set list) and the fourth data table (deleted index set list) are empty, which indicates that the current start has no change of any metadata and no new data is generated.
The database records index set snapshot data and detailed index set data, and all detailed data and change data are recorded. The developer only needs to modify index metadata with changes, so that the workload is greatly reduced. When the system is started, the file marked by the specific annotation is automatically scanned to generate corresponding index metadata for storage, so that the index metadata is always kept once with the latest index data of the system, and the problem of time difference caused by multi-system maintenance is avoided.
In one embodiment, a plurality of interfaces can be provided for other systems to acquire metadata related information, so that other systems can acquire accurate and detailed index metadata information, and the data can be used everywhere only by one definition. The interface comprises:
current index set snapshot information interface: the interface returns snapshot information of the current system index set. The system acquires the latest piece of data D1 in the index set snapshot table of the database table, queries the data in the detail table of the index set according to setId information in the index set list corresponding to the data D1 to acquire detailed index set information C1-CN, and assembles the data of the D1 and the C1-CN and returns the assembled data. The calling system acquires the current latest index set list information and detailed index data in each index set.
Index set snapshot information history interface: the interface is required to be transferred into the starting time T1 and T2, after the time format is checked, the interface queries the information of the time in the time period in the index set snapshot table of the database table, and returns the data to the calling interface after the data is extracted and formatted.
Index set snapshot information interface: the calling interface needs to be transmitted with the characteristic index set snapshot ID information ID0, inquires the data D0 with the ID of ID0 in the index set snapshot table of the database table, and if the data exists, inquires the data in the index set detail table according to setId information in the index set list of the data D0 to acquire detailed index set information C1-CN, and the data of D0 and C1-CN are returned after being assembled. The calling system will acquire the corresponding index set list information and detailed index data in each index set
Index set snapshot information comparison interface: the calling interface needs to transmit ID information ID1 and ID2 of the characteristic index set snapshot, the interface inquires data D1 and D2 with ID1 and ID2 in the index set snapshot table of the database table, if the data D1 and D2 exist, the data of the two index set lists are compared according to the setId information in the index set list corresponding to the data D1 and D2 to calculate new added, modified and deleted data of the index set list, and then the change of the description information of each index set and the new added, modified and deleted data of the internal index metadata are calculated, and the data is formatted and returned to the calling interface.
Referring to fig. 3, a schematic diagram of a preferred embodiment of an electronic device 1 according to the present invention is shown.
The electronic device 1 includes, but is not limited to: memory 11, processor 12, display 13, and network interface 14. The electronic device 1 is connected to a network through a network interface 14 to obtain the original data. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a global system for mobile communications (Global System of Mobile communication, GSM), wideband code division multiple access (Wideband Code Division Multiple Access, WCDMA), a 4G network, a 5G network, bluetooth (Bluetooth), wi-Fi, or a call network.
The memory 11 includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage 11 may be an internal storage unit of the electronic device 1, such as a hard disk or a memory of the electronic device 1. In other embodiments, the memory 11 may also be an external storage device of the electronic device 1, for example, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are equipped in the electronic device 1. Of course, the memory 11 may also comprise both an internal memory unit of the electronic device 1 and an external memory device. In this embodiment, the memory 11 is generally used to store an operating system installed in the electronic device 1 and various application software, such as program codes of the processing program 10 of the index data. Further, the memory 11 may be used to temporarily store various types of data that have been output or are to be output.
Processor 12 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 12 is typically used for controlling the overall operation of the electronic device 1, e.g. performing data interaction or communication related control and processing, etc. In this embodiment, the processor 12 is configured to execute the program code stored in the memory 11 or process data, such as the program code of the process program 10 running the index data.
The display 13 may be referred to as a display screen or a display unit. The display 13 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an Organic Light-Emitting Diode (OLED) touch device, or the like in some embodiments. The display 13 is used for displaying information processed in the electronic device 1 and for displaying a visual work interface, for example displaying the results of data statistics.
The network interface 14 may alternatively comprise a standard wired interface, a wireless interface, such as a WI-FI interface, which network interface 14 is typically used for establishing a communication connection between the electronic device 1 and other electronic devices.
Fig. 3 shows only the electronic device 1 with components 11-14 and the processing program 10 of the index data, but it is understood that not all shown components are required to be implemented, and that more or fewer components may alternatively be implemented.
Optionally, the electronic device 1 may further comprise a user interface, which may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), and a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an Organic Light-Emitting Diode (OLED) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device 1 and for displaying a visual user interface.
The electronic device 1 may further comprise Radio Frequency (RF) circuits, sensors and audio circuits etc., which are not described here.
In the above embodiment, the processor 12 may implement the following steps when executing the processing program 10 of the index data stored in the memory 11:
judging whether a database has a new index set, and storing relevant information corresponding to the new index set into a first data table and a second data table of a preset storage path when judging that the database is in the new index set;
When judging that the database does not have a newly added index set, judging whether the description information of the index set in the database is changed or not, judging whether index metadata of the index set is changed or not, and when judging that the description information of the index set or the index metadata information of the index set is changed, storing related information corresponding to the changed index set into a first data table and a third data table of the preset storage path;
judging whether the database has a deleted index set or not, if so, storing the related information of the deleted index set into a fourth data table of the preset storage path;
and when any one data table among the second data table, the third data table and the fourth data table is not empty, generating latest snapshot data and storing the latest snapshot data into a preset index set snapshot table.
The storage device may be the memory 11 of the electronic device 1, or may be another storage device communicatively connected to the electronic device 1.
For a detailed description of the above steps, please refer to the functional block diagram of the embodiment of the processing apparatus 100 for index data shown in fig. 2 and the flowchart of the embodiment of the processing method for index data shown in fig. 1.
Furthermore, the embodiment of the invention also provides a computer readable storage medium, which can be nonvolatile or volatile. The computer readable storage medium may be any one or any combination of several of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, etc. The computer readable storage medium includes a storage data area and a storage program area, the storage data area stores data created according to the use of the blockchain node, the storage program area stores a processing program 10 of index data, and the processing program 10 of the index data realizes the following operations when being executed by a processor:
judging whether a database has a new index set, and storing relevant information corresponding to the new index set into a first data table and a second data table of a preset storage path when judging that the database is in the new index set;
when judging that the database does not have a newly added index set, judging whether the description information of the index set in the database is changed or not, judging whether index metadata of the index set is changed or not, and when judging that the description information of the index set or the index metadata information of the index set is changed, storing related information corresponding to the changed index set into a first data table and a third data table of the preset storage path;
Judging whether the database has a deleted index set or not, if so, storing the related information of the deleted index set into a fourth data table of the preset storage path;
and when any one data table among the second data table, the third data table and the fourth data table is not empty, generating latest snapshot data and storing the latest snapshot data into a preset index set snapshot table.
The embodiment of the computer readable storage medium of the present invention is substantially the same as the embodiment of the method for processing index data, and will not be described herein.
It should be noted that, the foregoing reference numerals of the embodiments of the present invention are merely for describing the embodiments, and do not represent the advantages and disadvantages of the embodiments. And the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, apparatus, article or method that comprises the element.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, an electronic device, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (5)
1. A method for processing index data, the method comprising:
Judging whether a database has a new index set, and storing relevant information corresponding to the new index set into a first data table and a second data table of a preset storage path when judging that the database is in the new index set;
when judging that the database does not have a new index set, acquiring primary key information corresponding to the index set from an index set list corresponding to the index set, acquiring description information of snapshot data corresponding to the index set from an index detail table of the database based on the primary key information, judging whether the description information in the snapshot data is the same as the description information in the index set annotation information or not so as to judge whether the description information of the index set in the database is changed or not, and judging whether index metadata of the index set is changed or not;
when the description information of the index set is judged and the index metadata of the index set are unchanged, storing related information corresponding to the index set into a first data table of the preset storage path, wherein the related information comprises main key information recorded by the storage of the index set, english names and Chinese names of the index set;
or when judging that the description information of the index set or the index metadata information of the index set is changed, storing related information corresponding to the changed index set into a first data table and a third data table of the preset storage path, analyzing according to a predefined data structure to obtain the latest data after the index set is changed, and storing the latest data into an index detail table of the database;
Judging whether the database has a deleted index set or not, if so, storing the related information of the deleted index set into a fourth data table of the preset storage path;
and when any one data table among the second data table, the third data table and the fourth data table is not empty, generating latest snapshot data and storing the latest snapshot data into a preset index set snapshot table.
2. The method of processing index data according to claim 1, wherein after judging that there is a new index set in the database, the method further comprises:
and storing the index metadata of the newly-increased index set into an index detail table of the database.
3. An apparatus for processing index data, the apparatus comprising:
a first judging module: the method comprises the steps of judging whether a database has a new index set or not, and storing relevant information corresponding to the new index set into a first data table and a second data table of a preset storage path when the database is judged to be in the new index set;
and a second judging module: the method comprises the steps of obtaining primary key information corresponding to an index set from an index set list corresponding to the index set when judging that a newly added index set does not exist in a database, obtaining description information of snapshot data corresponding to the index set from an index detail table of the database based on the primary key information, judging whether the description information in the snapshot data is identical with the description information in the index set annotation information so as to judge whether the description information of the index set in the database is changed, judging whether index metadata of the index set is changed, storing related information corresponding to the index set into a first data table of a preset storage path when judging that the description information of the index set and the index metadata of the index set are not changed, wherein the related information comprises the primary key information of a storage record of the index set, an English name and a Chinese name of the index set, or storing the related information corresponding to the changed index set into a first data table of the preset storage path and a third data table of a latest data analysis table according to the latest defined data structure when judging that the description information of the index set or the index metadata of the index set is changed;
And a third judging module: the fourth data table is used for judging whether the database has the deleted index set or not, if so, the related information of the deleted index set is stored in the fourth data table to the preset storage path;
the generation module is used for: and generating latest snapshot data and storing the latest snapshot data into a preset index set snapshot table when any one of the second data table, the third data table and the fourth data table is not empty.
4. An electronic device, the electronic device comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a program executable by the at least one processor, the program being executable by the at least one processor to cause the at least one processor to perform the steps of:
judging whether a database has a new index set, and storing relevant information corresponding to the new index set into a first data table and a second data table of a preset storage path when judging that the database is in the new index set;
when judging that a new index set does not exist in the database, acquiring primary key information corresponding to the index set from an index set list corresponding to the index set, acquiring description information of snapshot data corresponding to the index set from an index detail table of the database based on the primary key information, judging whether the description information in the snapshot data is identical with the description information in the index set annotation information so as to judge whether the description information of the index set in the database is changed, judging whether index metadata of the index set is changed, and when judging that the description information of the index set and the index metadata of the index set are not changed, storing related information corresponding to the index set into a first data table of the preset storage path, wherein the related information comprises the primary key information of the index set storage record, the English name and the Chinese name of the index set;
Or when judging that the description information of the index set or the index metadata information of the index set is changed, storing related information corresponding to the changed index set into a first data table and a third data table of the preset storage path, analyzing according to a predefined data structure to obtain the latest data after the index set is changed, and storing the latest data into an index detail table of the database;
judging whether the database has a deleted index set or not, if so, storing the related information of the deleted index set into a fourth data table of the preset storage path;
and when any one data table among the second data table, the third data table and the fourth data table is not empty, generating latest snapshot data and storing the latest snapshot data into a preset index set snapshot table.
5. A computer-readable storage medium storing a processing program of index data, which when executed by a processor, implements the method of processing index data according to claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011444448.4A CN112465630B (en) | 2020-12-11 | 2020-12-11 | Index data processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011444448.4A CN112465630B (en) | 2020-12-11 | 2020-12-11 | Index data processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112465630A CN112465630A (en) | 2021-03-09 |
| CN112465630B true CN112465630B (en) | 2024-03-26 |
Family
ID=74800615
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011444448.4A Active CN112465630B (en) | 2020-12-11 | 2020-12-11 | Index data processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112465630B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107895049A (en) * | 2017-12-05 | 2018-04-10 | 泰康保险集团股份有限公司 | Data processing method and device, computer-readable recording medium, electronic equipment |
| CN110019267A (en) * | 2017-11-21 | 2019-07-16 | 中国移动通信有限公司研究院 | A kind of metadata updates method, apparatus, system, electronic equipment and storage medium |
| CN110321383A (en) * | 2019-05-20 | 2019-10-11 | 平安普惠企业管理有限公司 | Big data platform method of data synchronization, device, computer equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10860607B2 (en) * | 2018-07-27 | 2020-12-08 | EMC IP Holding Company LLC | Synchronization of metadata-based system snapshots with a state of user data |
-
2020
- 2020-12-11 CN CN202011444448.4A patent/CN112465630B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110019267A (en) * | 2017-11-21 | 2019-07-16 | 中国移动通信有限公司研究院 | A kind of metadata updates method, apparatus, system, electronic equipment and storage medium |
| CN107895049A (en) * | 2017-12-05 | 2018-04-10 | 泰康保险集团股份有限公司 | Data processing method and device, computer-readable recording medium, electronic equipment |
| CN110321383A (en) * | 2019-05-20 | 2019-10-11 | 平安普惠企业管理有限公司 | Big data platform method of data synchronization, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112465630A (en) | 2021-03-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110019486B (en) | Data acquisition method, device, equipment and storage medium | |
| CN108427705B (en) | Electronic device, distributed system log query method and storage medium | |
| CN110457363B (en) | Query method, device and storage medium based on distributed database | |
| CN111563368A (en) | Report generation method and device, computer equipment and storage medium | |
| CN109471857B (en) | SQL statement-based data modification method, device and storage medium | |
| CN109543891B (en) | Method and apparatus for establishing capacity prediction model, and computer-readable storage medium | |
| CN111367925A (en) | Data dynamic real-time updating method, device and storage medium | |
| CN110019038B (en) | File reference analysis method, device, equipment and computer readable storage medium | |
| CN108415998B (en) | Application dependency relationship updating method, terminal, device and storage medium | |
| CN113448862B (en) | Software version testing method and device and computer equipment | |
| CN112416957B (en) | Data increment updating method and device based on data model layer and computer equipment | |
| CN110806866A (en) | Generation method and device of front-end management system | |
| CN111767262A (en) | Log display method, device, equipment and storage medium | |
| CN115617780A (en) | Data import method, device, equipment and storage medium | |
| CN111723077A (en) | Data dictionary maintenance method and device and computer equipment | |
| CN112364187B (en) | Automobile accessory database building method, device and equipment based on big data | |
| CN112465630B (en) | Index data processing method, device, equipment and storage medium | |
| CN110955450B (en) | Attribution statistical method, system and storage medium of application package file | |
| CN112463549A (en) | Auditing method, device and equipment of cloud platform and computer readable storage medium | |
| CN111416857A (en) | Client crash processing method, device, system, equipment and storage medium | |
| CN113535206B (en) | Multi-version code upgrading method and system | |
| CN112988786B (en) | Database maintenance method and device | |
| CN110704523B (en) | Data export method, device, equipment and computer readable storage medium | |
| CN109902094B (en) | Log preservation method, device and computer readable storage medium | |
| CN111343502B (en) | Video processing method, electronic device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |