CN116521713A - Data query method, device, equipment and storage medium - Google Patents
Data query method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN116521713A CN116521713A CN202310805159.XA CN202310805159A CN116521713A CN 116521713 A CN116521713 A CN 116521713A CN 202310805159 A CN202310805159 A CN 202310805159A CN 116521713 A CN116521713 A CN 116521713A
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- parameters
- interval
- storing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 230000015654 memory Effects 0.000 claims description 27
- 238000012545 processing Methods 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 16
- 238000004458 analytical method Methods 0.000 claims description 2
- 238000007781 pre-processing Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 9
- 230000006872 improvement Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 229920001296 polysiloxane Polymers 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000010979 ruby Substances 0.000 description 1
- 229910001750 ruby Inorganic materials 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The specification discloses a method, a device, equipment and a storage medium for data query, which can save storage parameters of data as metadata by preprocessing data of each data interval in a data table, so that each data only needs to save corresponding mantissas, and therefore, the size relation between all data contained in the data interval and query condition values can be determined directly by comparing the storage parameters for the data interval with the same storage parameters of all the saved data, and further, query efficiency when data query operation is performed on Decimal type data according to conditions can be conveniently improved.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for querying data.
Background
At present, in the technical fields of big data, artificial intelligence, internet of things and the like, the processing of data is a very basic requirement, and in order to meet the data processing requirement, a database system provides various different data types for developers to use, so that the developers can process various different types of data, for example: the int data type is used to store integer data, the double data type is used to store floating point data, etc.
The database system allocates memories of different sizes for data of different data types, wherein the memory size to which the data of each data type is allocated is fixed, which makes the stored data of different data types limited in scope, and when the stored data is large, overflow situation occurs, so that data processing is abnormal. For example: a short type is usually allocated with two bytes of storage space, and the data range of integers that can be stored in the two bytes of storage space is-32768 to +32767, if the number +32768 is stored in the short type variable, data overflow occurs, and the number actually stored is-32768, so that data processing is abnormal.
However, due to the extremely high demands on the accuracy of data in some specific fields, there is provided a Decimal type (i.e., high-accuracy Decimal number) in a database system, which can meet the high accuracy demands on data by storing the data in a larger memory space and by converting the data into a form of sign, radix, exponent, mantissa. For example: if the data 123123456789.123456789 needs to be stored, the number can be split into 123×10, 9, 1+123456789 (10, 9), 0+123456789 (10, 9), and 1, where "123" and two "123456789" are mantissas, "10, 9" is a radix, "1", "0", and "1" are exponents, and the sign is positive.
However, due to the special storage mode of the Decimal data, the query efficiency is low when the data query operation is performed on the Decimal data according to the conditions.
Disclosure of Invention
The present disclosure provides a method, apparatus, device, and storage medium for querying data, so as to partially solve the problems in the prior art.
The technical scheme adopted in the specification is as follows:
the present specification provides a method of data querying, the data being stored in a database in the form of a sign parameter, an exponent parameter and a mantissa parameter, the method comprising:
acquiring a query statement;
analyzing the query statement to obtain a query condition value;
determining a condition parameter corresponding to the query condition value, wherein the condition parameter comprises: symbol parameters corresponding to the data to be queried and index parameters corresponding to the data to be queried;
for each data interval which is processed in advance, if the metadata corresponding to the data interval is determined, carrying out data query according to the metadata corresponding to the data interval and the condition parameters, wherein the metadata corresponding to the data interval is used for describing storage parameters corresponding to the data in the data interval, and the storage parameters comprise: symbol parameters corresponding to the data in the data interval and index parameters corresponding to the data in the data interval.
Optionally, preprocessing each data interval specifically includes:
and if the storage parameters corresponding to the data in the data interval are the same, the storage space for storing the storage parameters of each data in the data interval is emptied, and the storage parameters are used as metadata corresponding to the data interval to be stored.
Optionally, preprocessing each data interval specifically includes:
and if the storage parameters corresponding to the data in the data interval are different according to each data interval, clearing a storage space for storing the storage parameters corresponding to the data according to each data in the data interval, and storing the storage parameters corresponding to the data in the storage space for storing at least part of mantissa parameters of the data.
Optionally, storing the storage parameter corresponding to the data in the storage space storing at least part of the mantissa parameter of the data specifically includes:
when determining that a designated byte in a storage space storing at least part of mantissa parameters of the data is unoccupied, storing the storage parameters corresponding to the data in the designated byte.
Optionally, storing the storage parameters corresponding to the data in the storage space storing at least part of the mantissa parameters of the data specifically includes:
When determining that the designated bytes in the storage space storing at least part of the mantissa parameters of the data are occupied, expanding the storage space storing the at least part of the mantissa parameters, and storing the storage parameters corresponding to the data in the expanded storage space.
Optionally, the method further comprises:
for each data interval which is processed in advance, if the data interval does not correspond to metadata, determining a reference value used for representing the numerical value of the data as the reference value corresponding to the data according to each data in the data interval, wherein the reference value corresponding to the data is determined by the storage parameter of the data stored in the storage space for storing at least part of mantissa parameters of the data and the at least part of mantissa parameters;
and according to the query condition value and the reference value corresponding to each data in the data interval, carrying out data query from the data interval.
Optionally, the data stored in the database includes: decimal type data.
The present specification provides an apparatus for data querying, including:
the acquisition module is used for acquiring the query statement;
the analysis module is used for analyzing the query statement to obtain a query condition value;
The determining module is configured to determine a condition parameter corresponding to the query condition value, where the condition parameter includes: symbol parameters corresponding to the data to be queried and index parameters corresponding to the data to be queried;
the query module is configured to, for each data interval that is processed in advance, if it is determined that metadata corresponds to the data interval, perform data query according to the metadata corresponding to the data interval and the condition parameter, where the metadata corresponding to the data interval is used to describe a storage parameter corresponding to data in the data interval, where the storage parameter includes: symbol parameters corresponding to the data in the data interval and index parameters corresponding to the data in the data interval.
Optionally, the apparatus further comprises: a processing module;
the processing module is used for clearing a storage space for storing the storage parameter of each data in the data interval and storing the storage parameter as metadata corresponding to the data interval if the storage parameters corresponding to the data in the data interval are identical.
Optionally, the apparatus further comprises: a processing module;
the processing module is configured to, for each data interval, empty a storage space for storing a storage parameter corresponding to data in the data interval if it is determined that the storage parameters corresponding to the data in the data interval are different, and store the storage parameter corresponding to the data in the storage space for storing at least part of mantissa parameters of the data.
Optionally, the processing module is specifically configured to store, when it is determined that a specified byte in a storage space storing at least part of the mantissa parameter of the data is unoccupied, a storage parameter corresponding to the data in the specified byte.
Optionally, the processing module is specifically configured to, when determining that a specified byte in a storage space storing at least a part of the mantissa parameters of the data is occupied, expand the storage space storing the at least part of the mantissa parameters, and store the storage parameters corresponding to the data in the expanded storage space.
Optionally, the query module is specifically configured to, for each data interval that is processed in advance, determine, for each data in the data interval, a reference value for characterizing a value of the data, where the reference value corresponding to the data is determined by a storage parameter of the data stored in a storage space storing at least part of mantissa parameters of the data and the at least part of mantissa parameters, if it is determined that the data interval does not correspond to metadata; and according to the query condition value and the reference value corresponding to each data in the data interval, carrying out data query from the data interval.
The present specification provides a computer readable storage medium storing a computer program which when executed by a processor implements the method of data querying described above.
The present specification provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing a method of data querying as described above when executing the program.
The above-mentioned at least one technical scheme that this specification adopted can reach following beneficial effect:
in the data query method provided in the specification, a query statement is obtained, the query statement is analyzed to obtain a query condition value, and a condition parameter corresponding to the query condition value is determined, wherein the condition parameter comprises: for each data interval processed in advance, if metadata corresponding to the data interval is determined, performing data query according to the metadata corresponding to the data interval and the condition parameters, where the metadata corresponding to the data interval is used for describing storage parameters corresponding to the data in the data interval, and the storage parameters include: symbol parameters corresponding to the data in the data interval and index parameters corresponding to the data in the data interval.
According to the method, the data of each data interval in the data table can be preprocessed, and the storage parameters of the data can be stored as metadata, so that each data only needs to store the corresponding mantissa, and therefore the size relation between all the data contained in the data interval and the query condition value can be determined directly by comparing the storage parameters for the data interval with the same storage parameters of all the stored data, and further the query efficiency in the process of carrying out data query operation on the data of the Decimal type according to the conditions can be improved conveniently.
Drawings
The accompanying drawings, which are included to provide a further understanding of the specification, illustrate and explain the exemplary embodiments of the present specification and their description, are not intended to limit the specification unduly. In the drawings:
FIG. 1 is a flow chart of a method of querying data provided in the present specification;
FIG. 2 is a schematic diagram of the storage of Decimal type data provided in the present specification;
FIG. 3 is a schematic diagram of the storage of another Decimal-type data provided in this specification;
FIG. 4 is a schematic diagram of an apparatus for data querying provided in the present specification;
fig. 5 is a schematic diagram of an electronic device corresponding to fig. 1 provided in the present specification.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the present specification more apparent, the technical solutions of the present specification will be clearly and completely described below with reference to specific embodiments of the present specification and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present specification. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.
The following describes in detail the technical solutions provided by the embodiments of the present specification with reference to the accompanying drawings.
Fig. 1 is a flow chart of a method for querying data provided in the present specification, which includes the following steps:
s100: a query statement is obtained.
In the specification, when a user needs to inquire data, the user can input an inquiry statement through a client provided by the service platform and send the input inquiry statement to the service platform, so that the service platform responds to the inquiry statement to inquire the data after acquiring the inquiry statement.
It should be noted that, in the database, data is stored in a form of a table, and each column represents a field, and each field corresponds to an attribute, for example: in the table for storing the information of the bank user, the information of the user name, age, identification card number, account balance and the like of the user needs to be stored, wherein the user name, the age, the identification card number and the account balance are respectively one attribute of the user, namely, are respectively one field.
Further, in the table storing data, each row corresponds to an element of an object, that is, an attribute value of an object, for example: "Mr. tense", "28", "10011000101", "200", where "Mr. tense", "28", "10011000101", "200" are each an attribute value of Mr. tense of the user.
Further, in the table storing data, each column has a corresponding data type, for example: the data type of the column of the user name can be a character Char type, and the attribute of the account balance has strict requirements on the precision of the data, so the data type of the column of the account balance is usually a Decimal type.
When the database stores a piece of Decimal data, the data of Decimal type is usually split into a plurality of combinations of mantissas, radix and exponent and sign parameters for storage, and the following formula can be referred to specifically:
in the above-mentioned formula(s),data for representing a Decimal type, < >>、/>、/>I.e. mantissa parameter, ">I.e. the radix, the value b of which is typically the power of 10 to the nth powerThe common power of 10 to 9 is 0, -1, -n being the exponent parameter, the sign parameter being either 1 or 0 (i.e. positive or negative, which can be expressed as positive by 1 and negative by 0).
For example: for one Decimal type of data 123123456789.123456789, it can be split into:where b is the power of 10 to 9, "123", "123456789", "123456789" are mantissa parameters, e (i.e., 1, 0, -1) are exponent parameters, and symbol parameters are 1.
It should be noted that, in splitting a decmal type of data, nine bits are split once because, when storing, one byte can store eight binary values, so nine bits in decimal require four bytes to store (if the radix is smaller, fewer bytes may be used to store each mantissa, for example, two bytes are used to store mantissa), in addition, when storing a decmal type of data, the database typically stores mantissa parameters of a decmal type of data in an array (for example, an array composed of three numbers of "123", "123459", "123456789") and separately divides the sign parameters and exponent parameters of a decmal type of data into one byte to store, as shown in fig. 2.
FIG. 2 is a schematic diagram of the storage of Decimal type data provided in this specification.
In fig. 2, each square represents a byte, wherein a white square represents a byte in which data is not stored, a light gray square represents a byte in which a mantissa parameter is stored, a dark gray square represents a byte in which a sign parameter and an exponent parameter are stored, wherein the four squares from left to right are used to store a mantissa array corresponding to the mantissa (converted into what is binary, so that two bytes are occupied), so that only two bytes are occupied, the four squares from fifth to eighth store a mantissa array corresponding to the mantissa of the integer part, so that four bytes are occupied, the four squares from ninth to twelfth store a mantissa array corresponding to the mantissa of the Decimal part 123459, and the thirteenth square is used to store a storage parameter of the data of the Decimal type 123123456789.123456789.
From the above, it can be seen that, due to the special storage mode of the data of the Decimal type, the query efficiency is lower when the data query operation is performed on the data of the Decimal type according to the conditions, so that the service platform can preprocess a column of data of the Decimal type, which is stored in the database and is frequently read, so that the query efficiency when the user queries the column of data can be improved.
It should be noted that, in a table storing data, a column may store a large amount of data, for example: in the account balance column, account balance data of a plurality of users of the service platform may be stored, so when the service platform acquires and performs preprocessing, each data interval can be acquired from a column of data for multiple times according to actual requirements, that is, the service platform acquires at least part of data from a column of data as one data interval each time, and performs preprocessing on the data contained in the data interval.
Specifically, for each data interval, if it is determined that the storage parameters corresponding to the data in the data interval are the same, the server empties the storage space for storing the storage parameters of each data in the data interval, and stores the same storage parameters as metadata corresponding to the data interval, where the storage parameters include: symbol parameters corresponding to the data in the data interval and index parameters corresponding to the data in the data interval.
Further, for each data interval, if it is determined that the storage parameters corresponding to the data in the data interval are different, for each data in the data interval, a storage space for storing the storage parameters corresponding to the data is emptied, and the storage parameters corresponding to the data are stored in the storage space for storing at least part of mantissa parameters of the data.
Specifically, when the server determines that a specified byte in a storage space storing at least part of the mantissa parameter of the data is unoccupied, the storage parameter corresponding to the data is stored in the specified byte. When the server determines that the designated bytes in the storage space storing at least part of the mantissa parameters of the data are occupied, expanding the storage space storing at least part of the mantissa parameters, and storing the storage parameters corresponding to the data in the expanded storage space.
It should be noted that, as can be seen from the foregoing, when the database stores a Decimal data, the data may be generally divided into a plurality of mantissas for storage, such as "123", "123456789", and "123456789", where the database allocates a storage space with the same size for each mantissa for storing each mantissa, for example: while in a practical application scenario, the mantissa at the higher level (i.e., the mantissa with a higher number of bits, such as "123") will often use only a portion of the four bytes allocated for the mantissa, and not all of the four bytes, as shown in fig. 3.
FIG. 3 is a schematic diagram of another storage of Decimal type data provided in this specification.
As can be seen in connection with fig. 3, the server may take the highest byte in the storage space storing at least the higher mantissa parameter of the data as the specified byte. For example: assuming that four bytes a, b, c, d are allocated to the higher mantissa 123, where a and b are used to store the mantissa 123, c is occupied only after the mantissa occupies all of the storage space in the two bytes a and b, and d is occupied after c is occupied completely, where d is the highest byte, so in most cases, the highest byte d will tend to be idle, and therefore, the server may store the storage parameter corresponding to the data through the highest byte d.
After the server stores the storage parameters corresponding to the data in the storage space storing at least part of the mantissa parameters of the data, the storage space storing at least part of the mantissa parameters of the data can be filled. For example: assuming that the storage is a mantissa of 111, after the storage parameter 1 (sign is positive) and 1 (exponent is 1) corresponding to the data is stored, the storage parameter may be filled with 11000111, that is, the remaining storage space between the byte for storing the mantissa and the byte for storing the storage parameter is filled with 0, so that the storage parameter and the mantissa form a new number, and the new data is the reference value of the data, and since the sign bit and the exponent bit are sequentially used as the maximum bit of the new data, the larger the reference value of the data is, the larger the data is, that is, the reference value of the data may be used to represent the data value.
In the present specification, the execution body of the method for implementing the data query may refer to a designated device such as a server provided on a service platform, or may refer to a terminal device such as a desktop computer or a notebook computer, and for convenience of description, the method for implementing the data query provided in the present specification will be described below by taking the server as an example of the execution body.
S102: and analyzing the query statement to obtain a query condition value.
After the server obtains the query statement, the obtained query statement can be analyzed to obtain a query condition value, and then the data meeting the query condition can be queried from the preprocessed data according to the query condition value.
For example: when the user wants to inquire the names of the users with account balances larger than 50000 in all the users contained in the user table, 50000 can be used as an inquiry condition value, and then the server can inquire data meeting the inquiry condition from the preprocessed data.
S104: determining a condition parameter corresponding to the query condition value, wherein the condition parameter comprises: symbol parameters corresponding to the data to be queried and index parameters corresponding to the data to be queried.
The server may determine condition parameters corresponding to the query condition values, where the condition parameters include: symbol parameters corresponding to the data to be queried and index parameters corresponding to the data to be queried.
For example: if the user needs to query the account number of the user with account balance greater than 5000, the condition parameters corresponding to the query condition values may be symbol parameters and index parameters after 5000 is converted into data of the Decimal type, that is, the symbol parameters are 1 (integer), and the index parameters are 0.
For another example: if the user needs to query the account number of the user with the account balance greater than 5000 and less than 5000000000, the condition parameters corresponding to the query condition values may be the symbol parameters and the index parameters after converting 5000 into the data of the Decimal type, and the symbol parameters after converting 5000000000 into the data of the Decimal type, that is, the symbol parameters are 1 (as an integer), and the index parameters are 0 and 1.
S106: for each data interval which is processed in advance, if the metadata corresponding to the data interval is determined, carrying out data query according to the metadata corresponding to the data interval and the condition parameters, wherein the metadata corresponding to the data interval is used for describing storage parameters corresponding to the data in the data interval, and the storage parameters comprise: symbol parameters corresponding to the data in the data interval and index parameters corresponding to the data in the data interval.
The server may determine, for each data interval that is processed in advance, if metadata is corresponding to the data interval, a size relationship between metadata and a condition parameter of the data interval according to the metadata corresponding to the data interval, and determine, according to the size relationship between the metadata corresponding to the data interval and the condition parameter, whether data included in the data interval are all data to be queried, where the metadata corresponding to the data interval is used to describe a storage parameter corresponding to the data in the data interval, and the storage parameter includes: symbol parameters corresponding to the data in the data interval and index parameters corresponding to the data in the data interval.
If the storage parameters of all the data in the data section are the same, the storage parameters are stored as metadata, and therefore, when the size relationship between the metadata and the condition parameters of the data section is determined to satisfy the query condition in the query statement when the data query is performed for all the data in the data section in which the metadata exists, it is determined that the storage parameters of all the data in the data section satisfy the query condition.
For example: if the sign parameters of the storage parameters of all the data in the data interval are positive, and the query condition is that the number is greater than-100, the sign parameters in the condition parameters can be determined to be negative, and all the data in the data interval can be determined to be the number greater than-100.
For another example: when the sign parameters of all the data in the data interval are consistent with the sign parameters in the condition parameters, if the index parameters of all the data in the data interval are 1, and the query condition is a number greater than 100, the index parameters in the condition parameters can be determined to be 0, and if the index parameters of all the data in the data interval are determined to be a number greater than 100.
Of course, if the metadata corresponding to the data interval is consistent with the condition parameters, it may be determined whether each data in the data interval satisfies the query condition according to the mantissa parameters of all data in the data interval and the size relationship between the mantissa parameters corresponding to the data to be queried determined according to the query condition.
Further, the server may determine, for each data section that is processed in advance, a reference value for indicating the magnitude of the data value for each data in the data section if it is determined that the data section does not have metadata, and perform data query from the data section according to a magnitude relation between the reference value corresponding to each data in the data section and the query condition value as the reference value corresponding to the data.
In the foregoing, when determining the size relationship between the metadata corresponding to any one data interval and the condition parameter, the mantissa parameter of the data, the size relationship between the mantissa parameter corresponding to the data to be queried determined according to the query condition, and the size relationship between the reference value corresponding to each data in any one data interval and the query condition value, the server may determine the size relationship between the selected multiple pairs of data in parallel through a single instruction multiple data (Single Instruction Multiple Data, SIMD) instruction set preset in the CPU.
When comparing the size relationships between the data by the CPU, one data is usually read into the register according to one instruction, and then the data can be compared with the pre-read data for the condition, then the next data is read according to the next instruction, and then the data is compared with the pre-read data for the condition, and so on until all the data are compared. To improve the comparison efficiency, a plurality of data may be read at a time and compared, and when the size relationship between a plurality of selected data is determined in parallel by the SIMD instruction set, 256 bits of instruction set may be used as an example, 256 bits (32 bytes) of data may be processed at a time by one instruction execution, that is, 4 pieces of 8 bytes of data may be processed at a time, or 8 pieces of 4 bytes of data and 16 pieces of 2 bytes of data or 32 pieces of single bytes of data may be processed at a time, based on which, when the size relationship between a plurality of selected data is determined in parallel by the SIMD instruction set, a plurality of data may be read into a register according to one instruction, and further the size relationship may be compared with the data for conditional processing that is read in advance in parallel.
Therefore, when determining the size relationship between the selected pairs of data in parallel by the SIMD instruction set, the size of the data is required to be 8/4/2/1 byte, and as can be seen from the above, the server can make the storage space occupied by the data compressed from the originally required 3 bytes to 2 bytes, or from the originally required 5 bytes to 4 bytes, or from the originally required 9 bytes to 8 bytes, etc. by the above-described compression method of storing the storage parameters corresponding to the data into the storage space storing at least part of the mantissa parameters of the data, the requirement for the data when determining the size relationship between the selected pairs of data in parallel by the SIMD instruction set can be satisfied.
From the above, it can be seen that, by preprocessing the data of each data interval in the data table, the storage parameters of the data can be saved as metadata, so that each data only needs to save the corresponding mantissa, so that the size relationship between all the data contained in the data interval and the query condition value can be determined directly by comparing the storage parameters for the data interval with the same storage parameters of all the saved data, and further, the query efficiency when the data query operation is performed on the data of the explanatory type according to the condition can be conveniently improved.
The above method for querying data provided for one or more embodiments of the present disclosure further provides a corresponding device for querying data based on the same concept, as shown in fig. 4.
Fig. 4 is a schematic diagram of an apparatus for querying data provided in the present specification, including:
an obtaining module 401, configured to obtain a query statement;
the parsing module 402 is configured to parse the query statement to obtain a query condition value;
a determining module 403, configured to determine a condition parameter corresponding to the query condition value, where the condition parameter includes: symbol parameters corresponding to the data to be queried and index parameters corresponding to the data to be queried;
the query module 404 is configured to, for each data interval that is processed in advance, perform data query according to metadata corresponding to the data interval and the condition parameter if it is determined that metadata corresponding to the data interval is included, where the metadata corresponding to the data interval is used to describe a storage parameter corresponding to data in the data interval, and the storage parameter includes: symbol parameters corresponding to the data in the data interval and index parameters corresponding to the data in the data interval.
Optionally, the apparatus further comprises: a processing module 405;
The processing module is used for clearing a storage space for storing the storage parameter of each data in the data interval and storing the storage parameter as metadata corresponding to the data interval if the storage parameters corresponding to the data in the data interval are identical.
Optionally, the processing module 405 is specifically configured to, for each data interval, if it is determined that the storage parameters corresponding to the data in the data interval are different, empty a storage space for storing the storage parameters corresponding to the data for each data in the data interval, and store the storage parameters corresponding to the data in the storage space for storing at least part of the mantissa parameters of the data.
Optionally, the processing module 405 is specifically configured to store, when it is determined that a specified byte in the storage space storing at least part of the mantissa parameter of the data is unoccupied, a storage parameter corresponding to the data in the specified byte.
Optionally, the processing module 405 is specifically configured to, when determining that a specified byte in a storage space storing at least a part of the mantissa parameters of the data is occupied, expand the storage space storing the at least part of the mantissa parameters, and store the storage parameters corresponding to the data in the expanded storage space.
Optionally, the query module 404 is specifically configured to, for each data interval that is processed in advance, determine, for each data in the data interval, a reference value for characterizing the value of the data, where the reference value corresponding to the data is determined by a storage parameter of the data stored in a storage space storing at least part of mantissa parameters of the data and the at least part of mantissa parameters, if it is determined that the data interval does not correspond to metadata; and according to the query condition value and the reference value corresponding to each data in the data interval, carrying out data query from the data interval.
The present specification also provides a computer readable storage medium having stored thereon a computer program operable to perform a method of data querying as provided in fig. 1 above.
The present specification also provides a schematic structural diagram of an electronic device corresponding to fig. 1 shown in fig. 5. At the hardware level, as in fig. 5, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile storage, although it may include hardware required for other services. The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to realize the data query method of fig. 1. Of course, other implementations, such as logic devices or combinations of hardware and software, are not excluded from the present description, that is, the execution subject of the following processing flows is not limited to each logic unit, but may be hardware or logic devices.
In the 90 s of the 20 th century, improvements to one technology could clearly be distinguished as improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable Gate Array, FPGA)) is an integrated circuit whose logic function is determined by the programming of the device by a user. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented by using "logic compiler" software, which is similar to the software compiler used in program development and writing, and the original code before the compiling is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but not just one of the hdds, but a plurality of kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), lava, lola, myHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic controllers, and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in one or more software and/or hardware elements when implemented in the present specification.
It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable application detection device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable application detection device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable application detection device to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable application detection device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer implemented process such that the instructions which execute on the computer or other programmable device provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.
It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing is merely an example of the present specification and is not intended to limit the present specification. Various modifications and alterations to this specification will become apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, or the like, which are within the spirit and principles of the present description, are intended to be included within the scope of the claims of the present description.
Claims (16)
1. A method of data querying, the data being stored in a database in the form of a sign parameter, an exponent parameter, and a mantissa parameter, the method comprising:
acquiring a query statement;
analyzing the query statement to obtain a query condition value;
determining a condition parameter corresponding to the query condition value, wherein the condition parameter comprises: symbol parameters corresponding to the data to be queried and index parameters corresponding to the data to be queried;
For each data interval which is processed in advance, if the metadata corresponding to the data interval is determined, carrying out data query according to the metadata corresponding to the data interval and the condition parameters, wherein the metadata corresponding to the data interval is used for describing storage parameters corresponding to the data in the data interval, and the storage parameters comprise: symbol parameters corresponding to the data in the data interval and index parameters corresponding to the data in the data interval.
2. The method of claim 1, wherein each data interval is pre-processed, specifically comprising:
and if the storage parameters corresponding to the data in the data interval are the same, the storage space for storing the storage parameters of each data in the data interval is emptied, and the storage parameters are used as metadata corresponding to the data interval to be stored.
3. The method of claim 1, wherein each data interval is pre-processed, specifically comprising:
and if the storage parameters corresponding to the data in the data interval are different according to each data interval, clearing a storage space for storing the storage parameters corresponding to the data according to each data in the data interval, and storing the storage parameters corresponding to the data in the storage space for storing at least part of mantissa parameters of the data.
4. A method according to claim 3, wherein the storing of the corresponding storage parameters of the data in the storage space storing at least part of the mantissa parameters of the data comprises:
when determining that a designated byte in a storage space storing at least part of mantissa parameters of the data is unoccupied, storing the storage parameters corresponding to the data in the designated byte.
5. A method according to claim 3, wherein the storing of the corresponding storage parameters of the data in the storage space storing at least part of the mantissa parameters of the data comprises:
when determining that the designated bytes in the storage space storing at least part of the mantissa parameters of the data are occupied, expanding the storage space storing the at least part of the mantissa parameters, and storing the storage parameters corresponding to the data in the expanded storage space.
6. A method as claimed in claim 3, the method further comprising:
for each data interval which is processed in advance, if the data interval does not correspond to metadata, determining a reference value used for representing the numerical value of the data as the reference value corresponding to the data according to each data in the data interval, wherein the reference value corresponding to the data is determined by the storage parameter of the data stored in the storage space for storing at least part of mantissa parameters of the data and the at least part of mantissa parameters;
And according to the query condition value and the reference value corresponding to each data in the data interval, carrying out data query from the data interval.
7. The method of any one of claims 1-6, the data stored in the database comprising: decimal type data.
8. An apparatus for data querying, comprising:
the acquisition module is used for acquiring the query statement;
the analysis module is used for analyzing the query statement to obtain a query condition value;
the determining module is configured to determine a condition parameter corresponding to the query condition value, where the condition parameter includes: symbol parameters corresponding to the data to be queried and index parameters corresponding to the data to be queried;
the query module is configured to, for each data interval that is processed in advance, if it is determined that metadata corresponds to the data interval, perform data query according to the metadata corresponding to the data interval and the condition parameter, where the metadata corresponding to the data interval is used to describe a storage parameter corresponding to data in the data interval, where the storage parameter includes: symbol parameters corresponding to the data in the data interval and index parameters corresponding to the data in the data interval.
9. The apparatus of claim 8, the apparatus further comprising: a processing module;
The processing module is used for clearing a storage space for storing the storage parameter of each data in the data interval and storing the storage parameter as metadata corresponding to the data interval if the storage parameters corresponding to the data in the data interval are identical.
10. The apparatus of claim 8, the apparatus further comprising: a processing module;
the processing module is configured to, for each data interval, empty a storage space for storing a storage parameter corresponding to data in the data interval if it is determined that the storage parameters corresponding to the data in the data interval are different, and store the storage parameter corresponding to the data in the storage space for storing at least part of mantissa parameters of the data.
11. The apparatus according to claim 10, wherein the processing module is specifically configured to, when it is determined that a specified byte in the storage space storing at least part of the mantissa parameter of the data is unoccupied, store the storage parameter corresponding to the data in the specified byte.
12. The apparatus of claim 10, wherein the processing module is specifically configured to, when determining that a specified byte in a storage space storing at least a part of the mantissa parameter of the data is occupied, expand the storage space storing the at least part of the mantissa parameter, and store a storage parameter corresponding to the data in the expanded storage space.
13. The apparatus according to claim 10, wherein the query module is specifically configured to, for each data interval that is processed in advance, determine, for each data in the data interval, a reference value for characterizing a value of the data, where the reference value corresponds to the data, and the reference value corresponds to the data is determined by a storage parameter of the data stored in a storage space storing at least a part of mantissa parameters of the data and the at least a part of mantissa parameters, if it is determined that the data interval does not correspond to metadata; and according to the query condition value and the reference value corresponding to each data in the data interval, carrying out data query from the data interval.
14. The apparatus according to any one of claims 8 to 13, wherein the data stored in the database comprises: decimal type data.
15. A computer readable storage medium storing a computer program which, when executed by a processor, implements the method of any of the preceding claims 1-7.
16. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any of the preceding claims 1-7 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310805159.XA CN116521713B (en) | 2023-06-30 | 2023-06-30 | Data query method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310805159.XA CN116521713B (en) | 2023-06-30 | 2023-06-30 | Data query method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116521713A true CN116521713A (en) | 2023-08-01 |
| CN116521713B CN116521713B (en) | 2023-09-12 |
Family
ID=87405119
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310805159.XA Active CN116521713B (en) | 2023-06-30 | 2023-06-30 | Data query method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116521713B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140156670A1 (en) * | 2011-06-21 | 2014-06-05 | Nec Corporation | Numeric range search device, numeric range search method, and numeric range search program |
| US20170032024A1 (en) * | 2015-07-28 | 2017-02-02 | Alibaba Group Holding Limited | Information query |
| CN111581236A (en) * | 2020-04-02 | 2020-08-25 | 中国邮政储蓄银行股份有限公司 | Data query method and device |
| CN114064706A (en) * | 2021-11-15 | 2022-02-18 | 北京嘉和海森健康科技有限公司 | Data storage method and device and server |
| CN114326922A (en) * | 2021-11-23 | 2022-04-12 | 极芯通讯技术(南京)有限公司 | Data processing method and device |
| CN114328577A (en) * | 2021-12-15 | 2022-04-12 | 江苏银承网络科技股份有限公司 | Data query method and device |
| CN115952201A (en) * | 2023-02-10 | 2023-04-11 | 平安科技（深圳）有限公司 | Data query method, device, system and storage medium |
-
2023
- 2023-06-30 CN CN202310805159.XA patent/CN116521713B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140156670A1 (en) * | 2011-06-21 | 2014-06-05 | Nec Corporation | Numeric range search device, numeric range search method, and numeric range search program |
| US20170032024A1 (en) * | 2015-07-28 | 2017-02-02 | Alibaba Group Holding Limited | Information query |
| CN111581236A (en) * | 2020-04-02 | 2020-08-25 | 中国邮政储蓄银行股份有限公司 | Data query method and device |
| CN114064706A (en) * | 2021-11-15 | 2022-02-18 | 北京嘉和海森健康科技有限公司 | Data storage method and device and server |
| CN114326922A (en) * | 2021-11-23 | 2022-04-12 | 极芯通讯技术(南京)有限公司 | Data processing method and device |
| CN114328577A (en) * | 2021-12-15 | 2022-04-12 | 江苏银承网络科技股份有限公司 | Data query method and device |
| CN115952201A (en) * | 2023-02-10 | 2023-04-11 | 平安科技（深圳）有限公司 | Data query method, device, system and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116521713B (en) | 2023-09-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111581230B (en) | Method, equipment and medium for generating data analysis report | |
| CN110134668B (en) | Data migration method, device and equipment applied to blockchain | |
| CN110162796B (en) | News thematic creation method and device | |
| CN110263050B (en) | Data processing method, device, equipment and storage medium | |
| CN109656946B (en) | Multi-table association query method, device and equipment | |
| CN117828360A (en) | Model training method, model training device, model code generating device, storage medium and storage medium | |
| CN112182116B (en) | Data exploration method and device | |
| CN116521713B (en) | Data query method, device, equipment and storage medium | |
| CN112148746B (en) | Method, device, electronic device and storage medium for generating database table structure document | |
| CN107368281B (en) | Data processing method and device | |
| WO2024179154A1 (en) | Data processing method, apparatus, and device | |
| CN110096477B (en) | Data storage method and device | |
| CN116167431B (en) | Service processing method and device based on hybrid precision model acceleration | |
| CN107544753B (en) | Data processing method and device and server | |
| CN110019187B (en) | Data distribution method, device and equipment | |
| CN116010419A (en) | Method and device for creating unique index and optimizing logic deletion | |
| CN116384505A (en) | Data processing method and device, storage medium and electronic equipment | |
| CN111339117B (en) | Data processing method, device and equipment | |
| CN115391426A (en) | Data query method and device, storage medium and electronic equipment | |
| CN107645541B (en) | Data storage method and device and server | |
| CN111967767A (en) | Business risk identification method, device, equipment and medium | |
| CN111598219A (en) | Quantization method, device, equipment and medium for depth separable convolution | |
| CN116185307B (en) | Storage method and device of model data, storage medium and electronic equipment | |
| CN112434486B (en) | Automatic paging method, device, equipment and medium for device in PCB design | |
| CN116109008B (en) | Method and device for executing service, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |