CN112860737B - Data query method and device, electronic equipment and readable storage medium - Google Patents
Data query method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN112860737B CN112860737B CN202110267043.6A CN202110267043A CN112860737B CN 112860737 B CN112860737 B CN 112860737B CN 202110267043 A CN202110267043 A CN 202110267043A CN 112860737 B CN112860737 B CN 112860737B
- Authority
- CN
- China
- Prior art keywords
- field
- data
- query
- target
- application program
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012545 processing Methods 0.000 claims abstract description 10
- 230000004044 response Effects 0.000 claims description 13
- 238000012544 monitoring process Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 description 11
- 238000004891 communication Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to data processing, and discloses a data query method, which comprises the following steps: establishing an initial table corresponding to each application program; setting priority for each dimension field, determining the dimension field corresponding to each original field in each data source, and taking the set of the dimension fields corresponding to each original field as a field set; when the value of a specified field in a specified data source is changed, acquiring a target field set corresponding to the specified field, calculating a target value corresponding to each dimension field in the target field set, and updating the target value to a corresponding initial table to obtain a target table corresponding to each application program; and acquiring the name of the target application program and the name of the field to be inquired carried by the inquiry request, and inquiring the numerical value corresponding to the field to be inquired from the target table corresponding to the target application program to obtain an inquiry result. The invention also provides a data inquiry device, electronic equipment and a readable storage medium. The invention improves the data query efficiency.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a data query method and apparatus, an electronic device, and a readable storage medium.
Background
With the advent of the big data era, the data volume is more huge, and how to quickly and accurately query data is the current research focus. Currently, with the development of services and the increase of data volume, data to be queried is not only data in a single table, but may be related to data in multiple tables, a query logic needs to be executed serially, more than 10 interfaces may need to be called for one query, and the average response time of the interfaces is more than 1.5 seconds, thereby resulting in too long query time. Therefore, a data query method is needed to improve the query efficiency.
Disclosure of Invention
In view of the above, it is necessary to provide a data query method, aiming at improving the data query efficiency.
The data query method provided by the invention comprises the following steps:
acquiring first query data corresponding to each application program in a first time period from a first database, establishing a preset type initial table corresponding to each application program based on the first query data, and configuring corresponding query logic and operation logic for each dimension field in the initial table, wherein the operation logic comprises original field names participating in operation and data source names to which the original field names belong;
setting priority for each dimension field based on the query logic, determining the dimension field corresponding to each original field in each data source based on the operation logic, and taking the set of the dimension fields corresponding to each original field as a field set;
monitoring each data source in real time, when the value of a specified field in a specified data source is changed, acquiring a target field set corresponding to the specified field, calculating the target value corresponding to each dimension field in the target field set one by one based on the priority, updating the target value to a corresponding initial table, and obtaining the target table corresponding to each application program;
analyzing a query request sent by a user based on a client, acquiring a target application program name carried by the request and a field name to be queried of the target application program, and querying a numerical value corresponding to the field to be queried from a target table corresponding to the target application program to obtain a query result.
Optionally, the first query data includes names of queried fields corresponding to the respective application programs, queried times of the queried fields, and query response time, and the establishing of the initial table of the preset type corresponding to the respective application programs based on the first query data includes:
taking the inquired fields with the inquired times larger than a first threshold value and the inquired fields with the inquiry response time larger than a second threshold value as the corresponding dimension fields of each application program;
and filling the dimension fields into blank forms of preset types respectively according to the sequence of the inquired times from high to low to obtain initial forms corresponding to each application program.
Optionally, the query logic includes a dependency relationship and a dependency hierarchy of the query, and setting a priority for each dimension field based on the query logic includes:
the dimension fields are prioritized from high to low in the order of the dependency hierarchy from low to high, the lower the dependency hierarchy the higher the priority.
Optionally, the calculating a target value corresponding to each dimension field in the target field set one by one based on the priority includes:
receiving a data ciphertext and a standard information abstract value sent by a client to which the specified data source belongs;
acquiring a key corresponding to a client to which the specified data source belongs from a second database, and decrypting the data ciphertext by using the key to obtain changed data corresponding to the specified field;
and carrying out security check on the changed data based on the standard information abstract value, and when the security check is passed, sequentially calculating a target numerical value corresponding to each dimension field in the target field set according to the sequence of the priority from high to low.
Optionally, the performing security check on the changed data based on the standard information digest value includes:
calculating the information abstract value of the changed data, and judging whether the calculated information abstract value is consistent with the standard information abstract value;
and when the calculated information abstract value is consistent with the standard information abstract value, judging that the security check is passed.
Optionally, after determining whether the calculated message digest value is consistent with the standard message digest value, the method further includes:
and if the calculated information abstract value is inconsistent with the standard information abstract value, judging that the security verification fails, discarding the changed data, and sending early warning information to a preset client.
Optionally, the method further includes:
and acquiring second query data corresponding to each application program every other second time period, and updating the name and the numerical value of the dimension field in the target table corresponding to each application program based on the second query data to obtain an updated target table corresponding to each application program.
In order to solve the above problem, the present invention also provides a data query apparatus, including:
the system comprises an establishing module, a data source processing module and a data processing module, wherein the establishing module is used for acquiring first query data corresponding to each application program in a first time period from a first database, establishing a preset type initial table corresponding to each application program based on the first query data, and configuring corresponding query logic and operation logic for each dimension field in the initial table, and the operation logic comprises original field names participating in operation and data source names to which the original field names belong;
the determining module is used for setting priority for each dimension field based on the query logic, determining the dimension field corresponding to each original field in each data source based on the operation logic, and taking the set of the dimension fields corresponding to each original field as a field set;
the updating module is used for monitoring each data source in real time, when the value of a certain designated field in a certain designated data source is changed, acquiring a target field set corresponding to the designated field, calculating a target value corresponding to each dimension field in the target field set one by one based on the priority, updating the target value to a corresponding initial table, and acquiring a target table corresponding to each application program;
the query module is used for analyzing a query request sent by a user based on a client, acquiring a target application program name carried by the request and a field name to be queried of the target application program, and querying a numerical value corresponding to the field to be queried from a target table corresponding to the target application program to obtain a query result.
In order to solve the above problem, the present invention also provides an electronic device, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a data query program executable by the at least one processor, the data query program being executable by the at least one processor to enable the at least one processor to perform the data query method described above.
In order to solve the above problem, the present invention also provides a computer-readable storage medium having a data query program stored thereon, the data query program being executable by one or more processors to implement the above data query method.
Compared with the prior art, the method and the device have the advantages that the initial table of the preset type corresponding to each application program is established based on the first query data, and the step realizes that the multiple tables corresponding to the application programs are converted into the single table; secondly, setting priority for each dimension field based on query logic, determining the dimension field corresponding to each original field in each data source based on operation logic, and taking the set of the dimension fields corresponding to each original field as a field set; then, when the value of a certain designated field in a certain designated data source is changed, acquiring a target field set corresponding to the designated field, calculating a target value corresponding to each dimension field in the target field set one by one according to the priority, updating the target value to a corresponding initial table, and acquiring a target table corresponding to each application program, wherein the step realizes that the associated table data is correspondingly and automatically updated when the data source is changed, and ensures the accuracy of the data; and finally, acquiring the name of the target application program and the name of the field to be inquired carried by the inquiry request, and inquiring the numerical value corresponding to the field to be inquired from the target table corresponding to the target application program to obtain an inquiry result. Therefore, the invention improves the data query efficiency.
Drawings
Fig. 1 is a schematic flow chart of a data query method according to an embodiment of the present invention;
fig. 2 is a schematic block diagram of a data query apparatus according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device implementing a data query method according to an embodiment of the present invention;
the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
The invention provides a data query method. Fig. 1 is a schematic flow chart of a data query method according to an embodiment of the present invention. The method may be performed by an electronic device, which may be implemented by software and/or hardware.
In this embodiment, the data query method includes:
s1, acquiring first query data corresponding to each application program in a first time period from a first database, establishing a preset type initial table corresponding to each application program based on the first query data, and configuring corresponding query logic and operation logic for each dimension field in the initial table, wherein the operation logic comprises names of original fields participating in operation and names of data sources to which the original fields belong.
The query in this embodiment is a query for an application, for example, a shopping application for inventory information of a certain product.
The first query data includes names of queried fields corresponding to each application program, queried times of the queried fields, and query response time, and the establishing of the initial table of the preset type corresponding to each application program based on the first query data includes:
a11, using the inquired field with the inquired times larger than the first threshold value and the inquired field with the inquiry response time larger than the second threshold value as the corresponding dimension field of each application program;
and A12, filling the dimension fields into blank forms of preset types respectively according to the sequence of the inquired times from high to low to obtain initial forms corresponding to each application program.
In this embodiment, the initial table of the preset type is an elastic search table, and the elastic search table supports distributed, highly-extended, and highly-real-time search, and compared with other tables, the elastic search table can provide more efficient search query.
The fields which are frequently inquired and have long inquiry response time are used as dimension fields to be filled in the elastic search table, and then inquiry is directly carried out on the basis of the elastic search table, so that the inquiry efficiency can be greatly improved; meanwhile, the dimension field with more inquired times is placed in front of the elastic search table, so that the dimension field can be inquired more quickly.
In the embodiment, an elastic search table is established for each application program, and in other embodiments, an elastic search table can be respectively established for different service scenes according to service needs. For example, for an insurance application program, an elastic search table can be respectively established for vehicle insurance data, health insurance data and property insurance data according to the type of the risk; or respectively establishing an elastic search table for the insurance data, the renewal data and the claim settlement data according to the insurance process.
In this embodiment, the query logic and the operation logic corresponding to each dimension field of the application program may be analyzed and determined according to the program code corresponding to the application program.
S2, setting priority for each dimension field based on the query logic, determining the dimension field corresponding to each original field in each data source based on the operation logic, and taking the set of the dimension fields corresponding to each original field as a field set.
The query logic includes dependencies and dependency levels of queries, the setting a priority for each dimension field based on the query logic includes:
the dimension fields are prioritized from high to low in the order of the dependency hierarchy from low to high, the lower the dependency hierarchy the higher the priority.
For example, for insurance applications, basic information (a mobile phone number, a communication address, an identification number and the like) of a user can be directly inquired from a data source without depending on any intermediate data, so that the corresponding priority of the basic information is highest; the total annual claim amount needs to summarize data corresponding to each claim form of various risk types in the year, and the priority corresponding to the total annual claim amount is slightly lower.
The operation logic comprises an operation formula and fields participating in operation, the fields participating in operation comprise original fields and intermediate fields of operation in the data source, for example, for annual total claim amount, the claim amount corresponding to each claim slip is the original field, and the month claim amount and the year claim amount corresponding to each risk type are the intermediate fields. The operation logic based on the dimension field can determine the dimension field corresponding to each original field in each data source, and when the value of the original field changes, the value of each dimension field in the corresponding field set changes.
And S3, monitoring each data source in real time, when the value of a certain designated field in a certain designated data source is changed, acquiring a target field set corresponding to the designated field, calculating the target value corresponding to each dimension field in the target field set one by one based on the priority, updating the target value to a corresponding initial table, and acquiring the target table corresponding to each application program.
The calculating the target value corresponding to each dimension field in the target field set one by one based on the priority comprises:
b11, receiving a data ciphertext and a standard information digest value sent by the client to which the specified data source belongs;
b12, obtaining a key corresponding to the client to which the specified data source belongs from a second database, and decrypting the data ciphertext by using the key to obtain changed data corresponding to the specified field;
b13, performing security check on the changed data based on the standard information abstract value, and when the security check is passed, sequentially calculating a target numerical value corresponding to each dimension field in the target field set according to the sequence of the priority from high to low.
In this embodiment, to ensure the security of data transmission, the client to which the data source belongs may encrypt data before reporting the data, and may encrypt the data by using a symmetric encryption key or a private key in asymmetric encryption, and the second database stores a symmetric key or an asymmetric public key corresponding to the client to which each data source belongs, so as to decrypt a received data ciphertext.
When the target values corresponding to all the dimension fields in the target field set are calculated, calculation is performed according to the sequence from high to low in priority (namely the sequence from low to high in dependency hierarchy), the target values of the dimension fields without dependency relationships are updated earliest, then the target values of the dimension fields with lower dependency hierarchies are updated, the target values of the dimension fields with lower dependency hierarchies can be calculated as middle values by the dimension fields with higher dependency hierarchies, repeated calculation is avoided, and calculation efficiency is improved. For example, the annual claim total can be calculated by taking the calculated month claim amount as intermediate data, and repeated calculation according to the original data of each claim slip is avoided.
In this embodiment, the performing, on the basis of the standard information digest, security verification on the changed data includes:
c11, calculating the information abstract value of the changed data, and judging whether the calculated information abstract value is consistent with the standard information abstract value;
and C12, judging that the security check is passed when the calculated message abstract value is consistent with the standard message abstract value.
The information abstract value is an MD5 value, and the safety (not tampered) and integrity of the data ciphertext can be further ensured by comparing whether the information abstract value obtained through calculation is consistent with the standard information abstract value.
In this embodiment, after determining whether the calculated message digest value is consistent with the standard message digest value, the method further includes:
and if the calculated information abstract value is inconsistent with the standard information abstract value, judging that the security verification fails, discarding the changed data, and sending early warning information to a preset client.
S4, analyzing a query request sent by a user based on a client, acquiring a target application program name carried by the request and a field name to be queried of the target application program, and querying a numerical value corresponding to the field to be queried from a target table corresponding to the target application program to obtain a query result.
In the embodiment, when the value of a certain original field of the data source is changed, the value of the associated dimension field in the table is updated in real time, so that the accuracy of the data in the target table is ensured; during query, single-table query can be performed according to the target table, so that the condition that one query needs to wait for responses of a plurality of interfaces in the prior art is effectively avoided, complex query logic does not need to be executed, and the query efficiency is greatly improved.
In this embodiment, the method further includes:
and acquiring second query data corresponding to each application program every other second time period, and updating the name and the numerical value of the dimension field in the target table corresponding to each application program based on the second query data to obtain the updated target table corresponding to each application program.
For example, updating the dimension field names in the target table once a month may make the dimension fields in the target table more consistent with current query requirements.
According to the data query method provided by the embodiment of the invention, firstly, the initial table of the preset type corresponding to each application program is established based on the first query data, and the step realizes that the multiple tables corresponding to the application programs are converted into the single table; secondly, setting priority for each dimension field based on query logic, determining the dimension field corresponding to each original field in each data source based on operation logic, and taking the set of the dimension fields corresponding to each original field as a field set; then, when the value of a certain designated field in a certain designated data source is changed, acquiring a target field set corresponding to the designated field, calculating a target value corresponding to each dimension field in the target field set one by one according to the priority, updating the target value to a corresponding initial table, and acquiring a target table corresponding to each application program, wherein the step realizes that the associated table data is correspondingly and automatically updated when the data source is changed, and ensures the accuracy of the data; and finally, acquiring the name of the target application program and the name of the field to be inquired carried by the inquiry request, and inquiring the numerical value corresponding to the field to be inquired from the target table corresponding to the target application program to obtain an inquiry result. Therefore, the invention improves the data query efficiency.
Fig. 2 is a schematic block diagram of a data query apparatus according to an embodiment of the present invention.
The data query apparatus 100 according to the present invention may be installed in an electronic device. Depending on the implemented functions, the data query apparatus 100 may include a setup module 110, a determination module 120, an update module 130, and a query module 140. The module of the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and that can perform a fixed function, and that are stored in a memory of the electronic device.
In the present embodiment, the functions regarding the respective modules/units are as follows:
the establishing module 110 is configured to obtain first query data corresponding to each application program in a first time period from a first database, establish an initial table of a preset type corresponding to each application program based on the first query data, and configure corresponding query logic and operation logic for each dimension field in the initial table, where the operation logic includes an original field name participating in operation and a data source name to which the original field name belongs.
The query in this embodiment is a query for an application, for example, a shopping application for inventory information of a certain product.
The first query data includes names of queried fields corresponding to each application program, queried times of the queried fields, and query response time, and the establishing of the preset type initial table corresponding to each application program based on the first query data includes:
a21, using the inquired field with the inquired times larger than the first threshold value and the inquired field with the inquiry response time larger than the second threshold value as the corresponding dimension field of each application program;
and A22, filling the dimension fields into blank forms of preset types respectively according to the sequence of the inquired times from high to low to obtain initial forms corresponding to each application program.
In this embodiment, the initial table of the preset type is an elastic search table, and the elastic search table supports distributed, highly-extended, and highly-real-time search, and compared with other tables, the elastic search table can provide more efficient search query.
The fields which are frequently inquired and have long inquiry response time are used as dimension fields to be filled in the elastic search table, and then inquiry is directly carried out on the basis of the elastic search table, so that the inquiry efficiency can be greatly improved; meanwhile, the dimension field with more inquired times is placed in front of the elastic search table, so that the dimension field can be inquired more quickly.
In the embodiment, an elastic search table is established for each application program, and in other embodiments, an elastic search table can be respectively established for different service scenes according to service needs. For example, for an insurance application program, an elastic search table can be respectively established for vehicle insurance data, health insurance data and property insurance data according to the type of the risk; or respectively establishing an elastic search table for the insurance data, the renewal data and the claim settlement data according to the insurance process.
In this embodiment, the query logic and the operation logic corresponding to each dimension field of the application program may be analyzed and determined according to the program code corresponding to the application program.
A determining module 120, configured to set a priority for each dimension field based on the query logic, determine, based on the operation logic, a dimension field corresponding to each original field in each data source, and use a set of dimension fields corresponding to each original field as a field set.
The query logic includes dependencies and dependency levels of queries, the setting a priority for each dimension field based on the query logic includes:
the dimension fields are prioritized from high to low in the order of the dependency hierarchy from low to high, the lower the dependency hierarchy the higher the priority.
For example, for insurance applications, basic information (a mobile phone number, a communication address, an identification number and the like) of a user can be directly inquired from a data source without depending on any intermediate data, so that the corresponding priority of the basic information is highest; the total annual claim amount needs to summarize data corresponding to each claim form of various risk types in the year, and the priority corresponding to the total annual claim amount is slightly lower.
The operation logic comprises an operation formula and fields participating in operation, the fields participating in operation comprise original fields and intermediate fields of operation in the data source, for example, for annual total claim amount, the claim amount corresponding to each claim slip is the original field, and the month claim amount and the year claim amount corresponding to each risk type are the intermediate fields. The operation logic based on the dimension field can determine the dimension field corresponding to each original field in each data source, and when the value of the original field changes, the value of each dimension field in the corresponding field set changes.
The updating module 130 is configured to monitor each data source in real time, obtain a target field set corresponding to a specific field when a value of the specific field in a specific data source changes, calculate a target value corresponding to each dimension field in the target field set one by one based on the priority, update the target value to a corresponding initial table, and obtain a target table corresponding to each application program.
The calculating the target value corresponding to each dimension field in the target field set one by one based on the priority comprises:
b21, receiving a data ciphertext and a standard information digest value sent by the client to which the specified data source belongs;
b22, obtaining a key corresponding to the client to which the specified data source belongs from a second database, and decrypting the data ciphertext by using the key to obtain changed data corresponding to the specified field;
b23, performing security check on the changed data based on the standard information abstract value, and when the security check is passed, sequentially calculating a target numerical value corresponding to each dimension field in the target field set according to the sequence of the priority from high to low.
In this embodiment, to ensure the security of data transmission, the client to which the data source belongs may encrypt data before reporting the data, and may encrypt the data by using a symmetric encryption key or a private key in asymmetric encryption, and the second database stores a symmetric key or an asymmetric public key corresponding to the client to which each data source belongs, so as to decrypt a received data ciphertext.
When the target values corresponding to all the dimension fields in the target field set are calculated, calculation is performed according to the sequence from high to low in priority (namely the sequence from low to high in dependency hierarchy), the target values of the dimension fields without dependency relationships are updated earliest, then the target values of the dimension fields with lower dependency hierarchies are updated, the target values of the dimension fields with lower dependency hierarchies can be calculated as middle values by the dimension fields with higher dependency hierarchies, repeated calculation is avoided, and calculation efficiency is improved. For example, the annual claim total can be calculated by taking the calculated month claim amount as intermediate data, and repeated calculation according to the original data of each claim slip is avoided.
In this embodiment, the performing, on the basis of the standard information digest, security verification on the changed data includes:
c21, calculating the information abstract value of the changed data, and judging whether the calculated information abstract value is consistent with the standard information abstract value;
and C22, judging that the security check is passed when the calculated message abstract value is consistent with the standard message abstract value.
The information abstract value is an MD5 value, and the safety (not tampered) and integrity of the data ciphertext can be further ensured by comparing whether the information abstract value obtained through calculation is consistent with the standard information abstract value.
In this embodiment, after determining whether the calculated message digest value is consistent with the standard message digest value, the updating module 130 is further configured to:
and if the calculated information abstract value is inconsistent with the standard information abstract value, judging that the security verification fails, discarding the changed data, and sending early warning information to a preset client.
The query module 140 is configured to analyze a query request sent by a user based on a client, obtain a name of a target application program and a name of a field to be queried of the target application program, which are carried in the request, and query a value corresponding to the field to be queried from a target table corresponding to the target application program, so as to obtain a query result.
In the embodiment, when the value of a certain original field of the data source is changed, the value of the associated dimension field in the table is updated in real time, so that the accuracy of the data in the target table is ensured; during query, single-table query can be performed according to the target table, so that the condition that one query needs to wait for responses of a plurality of interfaces in the prior art is effectively avoided, complex query logic does not need to be executed, and the query efficiency is greatly improved.
In this embodiment, the query module 140 is further configured to:
and acquiring second query data corresponding to each application program every other second time period, and updating the name and the numerical value of the dimension field in the target table corresponding to each application program based on the second query data to obtain an updated target table corresponding to each application program.
For example, updating the dimension field names in the target table once a month may make the dimension fields in the target table more consistent with current query requirements.
Fig. 3 is a schematic structural diagram of an electronic device for implementing a data query method according to an embodiment of the present invention.
The electronic device 1 is a device capable of automatically performing numerical calculation and/or information processing according to a command set or stored in advance. The electronic device 1 may be a computer, or may be a single network server, a server group composed of a plurality of network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing, where cloud computing is one of distributed computing and is a super virtual computer composed of a group of loosely coupled computers.
In the present embodiment, the electronic device 1 includes, but is not limited to, a memory 11, a processor 12, and a network interface 13, which are communicatively connected to each other through a system bus, wherein the memory 11 stores a data query program 10, and the data query program 10 is executable by the processor 12. Fig. 3 shows only the electronic device 1 with the components 11-13 and the data query program 10, and it will be understood by those skilled in the art that the structure shown in fig. 3 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or some components may be combined, or a different arrangement of components.
The storage 11 includes a memory and at least one type of readable storage medium. The memory provides cache for the operation of the electronic equipment 1; the readable storage medium may be a non-volatile storage medium such as flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the readable storage medium may be an internal storage unit of the electronic device 1, such as a hard disk of the electronic device 1; in other embodiments, the non-volatile storage medium may also be an external storage device of the electronic device 1, such as a plug-in hard disk provided on the electronic device 1, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like. In this embodiment, the readable storage medium of the memory 11 is generally used for storing an operating system and various application software installed in the electronic device 1, for example, codes of the data query program 10 in an embodiment of the present invention are stored. Further, the memory 11 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 12 is generally configured to control the overall operation of the electronic device 1, such as performing control and processing related to data interaction or communication with other devices. In this embodiment, the processor 12 is configured to run the program code stored in the memory 11 or process data, for example, run the data query program 10.
The network interface 13 may comprise a wireless network interface or a wired network interface, and the network interface 13 is used for establishing a communication connection between the electronic device 1 and a client (not shown).
Optionally, the electronic device 1 may further include a user interface, the user interface may include a Display (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface may further include a standard wired interface and a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
The data query program 10 stored in the memory 11 of the electronic device 1 is a combination of instructions, which when executed in the processor 12, can implement:
acquiring first query data corresponding to each application program in a first time period from a first database, establishing a preset type initial table corresponding to each application program based on the first query data, and configuring corresponding query logic and operation logic for each dimension field in the initial table, wherein the operation logic comprises original field names participating in operation and data source names of the original field names;
setting priority for each dimension field based on the query logic, determining the dimension field corresponding to each original field in each data source based on the operation logic, and taking the set of the dimension fields corresponding to each original field as a field set;
monitoring each data source in real time, when the value of a specified field in a specified data source is changed, acquiring a target field set corresponding to the specified field, calculating the target value corresponding to each dimension field in the target field set one by one based on the priority, updating the target value to a corresponding initial table, and obtaining the target table corresponding to each application program;
analyzing a query request sent by a user based on a client, acquiring a target application program name carried by the request and a field name to be queried of the target application program, and querying a numerical value corresponding to the field to be queried from a target table corresponding to the target application program to obtain a query result.
Specifically, the processor 12 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the data query program 10, which is not described herein again.
Further, the integrated modules/units of the electronic device 1, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. The computer readable medium may be non-volatile or non-volatile. The computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
The computer-readable storage medium stores the data query program 10, and the data query program 10 may be executed by one or more processors, and the specific implementation of the computer-readable storage medium of the present invention is substantially the same as that of each embodiment of the data query method, and is not described herein again.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only intended to illustrate the technical solutions of the present invention and not to limit the same, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A method for data query, the method comprising:
acquiring first query data corresponding to each application program in a first time period from a first database, establishing a preset type initial table corresponding to each application program based on the first query data, and configuring corresponding query logic and operation logic for each dimension field in the initial table, wherein the operation logic comprises original field names participating in operation and data source names to which the original field names belong;
setting priority for each dimension field based on the query logic, determining the dimension field corresponding to each original field in each data source based on the operation logic, and taking the set of the dimension fields corresponding to each original field as a field set;
monitoring each data source in real time, when the value of a specified field in a specified data source is changed, acquiring a target field set corresponding to the specified field, calculating the target value corresponding to each dimension field in the target field set one by one based on the priority, updating the target value to a corresponding initial table, and obtaining the target table corresponding to each application program;
analyzing a query request sent by a user based on a client, acquiring a target application program name carried by the request and a field name to be queried of the target application program, and querying a numerical value corresponding to the field to be queried from a target table corresponding to the target application program to obtain a query result.
2. The data query method of claim 1, wherein the first query data includes a name of a queried field corresponding to each application, a queried number thereof, and query response time, and the establishing of the initial table of the preset type corresponding to each application based on the first query data includes:
taking the inquired fields with the inquired times larger than a first threshold value and the inquired fields with the inquiry response time larger than a second threshold value as the corresponding dimension fields of each application program;
and filling the dimension fields into blank forms of preset types respectively according to the sequence of the inquired times from high to low to obtain initial forms corresponding to each application program.
3. The data query method of claim 1, wherein the query logic includes dependencies and dependency hierarchies for queries, the prioritizing each dimension field based on the query logic comprising:
the dimension fields are prioritized from high to low in the order of the dependency hierarchy from low to high, the lower the dependency hierarchy the higher the priority.
4. The data query method of claim 1, wherein said calculating a target value for each dimension field in the target field set one by one based on the priority comprises:
receiving a data ciphertext and a standard information abstract value sent by a client to which the specified data source belongs;
acquiring a key corresponding to a client to which the specified data source belongs from a second database, and decrypting the data ciphertext by using the key to obtain changed data corresponding to the specified field;
and carrying out security check on the changed data based on the standard information abstract value, and when the security check is passed, sequentially calculating a target numerical value corresponding to each dimension field in the target field set according to the sequence of the priority from high to low.
5. The data query method of claim 4, wherein said performing a security check on said altered data based on said standard message digest value comprises:
calculating the information abstract value of the changed data, and judging whether the calculated information abstract value is consistent with the standard information abstract value;
and when the calculated information abstract value is consistent with the standard information abstract value, judging that the security check is passed.
6. The data query method of claim 5, wherein after determining whether the computed message digest value is consistent with the standard message digest value, the method further comprises:
and if the calculated information abstract value is inconsistent with the standard information abstract value, judging that the security verification fails, discarding the changed data, and sending early warning information to a preset client.
7. The data query method of claim 1, wherein the method further comprises:
and acquiring second query data corresponding to each application program every other second time period, and updating the name and the numerical value of the dimension field in the target table corresponding to each application program based on the second query data to obtain the updated target table corresponding to each application program.
8. A data query apparatus, characterized in that the apparatus comprises:
the system comprises an establishing module, a data source processing module and a data processing module, wherein the establishing module is used for acquiring first query data corresponding to each application program in a first time period from a first database, establishing a preset type initial table corresponding to each application program based on the first query data, and configuring corresponding query logic and operation logic for each dimension field in the initial table, and the operation logic comprises original field names participating in operation and data source names to which the original field names belong;
the determining module is used for setting priority for each dimension field based on the query logic, determining the dimension field corresponding to each original field in each data source based on the operation logic, and taking the set of the dimension fields corresponding to each original field as a field set;
the updating module is used for monitoring each data source in real time, when the value of a specified field in a specified data source is changed, acquiring a target field set corresponding to the specified field, calculating a target value corresponding to each dimension field in the target field set one by one based on the priority, updating the target value to a corresponding initial table, and acquiring a target table corresponding to each application program;
the query module is used for analyzing a query request sent by a user based on a client, acquiring a target application program name carried by the request and a field name to be queried of the target application program, and querying a numerical value corresponding to the field to be queried from a target table corresponding to the target application program to obtain a query result.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a data query program executable by the at least one processor, the data query program being executable by the at least one processor to enable the at least one processor to perform the data query method of any one of claims 1 to 7.
10. A computer-readable storage medium having stored thereon a data query program executable by one or more processors to implement the data query method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110267043.6A CN112860737B (en) | 2021-03-11 | 2021-03-11 | Data query method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110267043.6A CN112860737B (en) | 2021-03-11 | 2021-03-11 | Data query method and device, electronic equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112860737A CN112860737A (en) | 2021-05-28 |
| CN112860737B true CN112860737B (en) | 2022-08-12 |
Family
ID=75994182
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110267043.6A Active CN112860737B (en) | 2021-03-11 | 2021-03-11 | Data query method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112860737B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113705184B (en) * | 2021-09-01 | 2023-09-22 | 同盾科技有限公司 | Custom report generation method and device, storage medium and electronic equipment |
| CN115314751B (en) * | 2022-08-08 | 2024-07-23 | 北京达佳互联信息技术有限公司 | Data processing method, device, electronic equipment and storage medium |
| CN117151555B (en) * | 2023-11-01 | 2024-02-02 | 青岛文达通科技股份有限公司 | Smart city service system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7657515B1 (en) * | 2007-03-30 | 2010-02-02 | Alexa Internet | High efficiency document search |
| US8447774B1 (en) * | 2004-11-23 | 2013-05-21 | Progress Software Corporation | Database-independent mechanism for retrieving relational data as XML |
| CN105512200A (en) * | 2015-11-26 | 2016-04-20 | 华为技术有限公司 | Distributed database processing method and device |
| CN110457346A (en) * | 2019-07-05 | 2019-11-15 | 中国平安财产保险股份有限公司 | Data query method, apparatus and computer readable storage medium |
| CN111125099A (en) * | 2019-12-10 | 2020-05-08 | 苏宁云计算有限公司 | Method and device for processing associated data based on Druid broad table |
| CN111782923A (en) * | 2020-06-24 | 2020-10-16 | 平安科技（深圳）有限公司 | Data query method and device, electronic equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2000235513A1 (en) * | 2000-03-31 | 2001-10-15 | Kapow Aps | Method of retrieving attributes from at least two data sources |
| US7739309B2 (en) * | 2004-09-30 | 2010-06-15 | Microsoft Corporation | Method, system, and computer-readable medium for merging data from multiple data sources for use in an electronic document |
| US7797312B2 (en) * | 2006-04-06 | 2010-09-14 | International Business Machines Corporation | Database query processing method and system |
| US20130173662A1 (en) * | 2012-01-03 | 2013-07-04 | International Business Machines Corporation | Dependency based prioritization of sub-queries and placeholder resolution |
-
2021
- 2021-03-11 CN CN202110267043.6A patent/CN112860737B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8447774B1 (en) * | 2004-11-23 | 2013-05-21 | Progress Software Corporation | Database-independent mechanism for retrieving relational data as XML |
| US7657515B1 (en) * | 2007-03-30 | 2010-02-02 | Alexa Internet | High efficiency document search |
| CN105512200A (en) * | 2015-11-26 | 2016-04-20 | 华为技术有限公司 | Distributed database processing method and device |
| CN110457346A (en) * | 2019-07-05 | 2019-11-15 | 中国平安财产保险股份有限公司 | Data query method, apparatus and computer readable storage medium |
| CN111125099A (en) * | 2019-12-10 | 2020-05-08 | 苏宁云计算有限公司 | Method and device for processing associated data based on Druid broad table |
| CN111782923A (en) * | 2020-06-24 | 2020-10-16 | 平安科技（深圳）有限公司 | Data query method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 干货 | Elasticsearch多表关联设计指南;@ 小浩;《https://www.cnblogs.com/xiohao/p/12774882.html》;20200425;第1-5页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112860737A (en) | 2021-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112860737B (en) | Data query method and device, electronic equipment and readable storage medium | |
| CN112445854B (en) | Multi-source service data real-time processing method, device, terminal and storage medium | |
| CN111400308B (en) | Processing method of cache data, electronic device and readable storage medium | |
| CN112416728A (en) | Buried point data acquisition method and device, client device and readable storage medium | |
| CN113342498A (en) | Concurrent request processing method, device, server and storage medium | |
| CN112596932A (en) | Service registration and interception method and device, electronic equipment and readable storage medium | |
| CN112948418A (en) | Dynamic query method, device, equipment and storage medium | |
| CN111814045B (en) | Data query method, device, electronic equipment and storage medium | |
| CN112506486A (en) | Search system establishing method and device, electronic equipment and readable storage medium | |
| CN114185895A (en) | Data import and export method and device, electronic equipment and storage medium | |
| CN110309496B (en) | Data summarizing method, electronic device and computer readable storage medium | |
| CN111159183B (en) | Report generation method, electronic device and computer readable storage medium | |
| CN112699142A (en) | Cold and hot data processing method and device, electronic equipment and storage medium | |
| CN112631731A (en) | Data query method and device, electronic equipment and storage medium | |
| CN111723077A (en) | Data dictionary maintenance method and device and computer equipment | |
| CN112527280B (en) | Interface display method, device, server and readable storage medium | |
| CN115409008A (en) | Chart generation method and device, electronic equipment and readable storage medium | |
| CN113342822A (en) | Data query method and device based on multi-type database, electronic equipment and medium | |
| CN113094248A (en) | User behavior data analysis method and device, electronic equipment and medium | |
| WO2019071899A1 (en) | Electronic device, vehicle data import method and storage medium | |
| CN113590703B (en) | ES data importing method and device, electronic equipment and readable storage medium | |
| CN115086047B (en) | Interface authentication method and device, electronic equipment and storage medium | |
| CN112286703B (en) | User classification method and device, client device and readable storage medium | |
| CN113364848B (en) | File caching method and device, electronic equipment and storage medium | |
| CN112182598A (en) | Public sample ID identification method, device, server and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |