CN112069175A - Data query method and device and electronic equipment - Google Patents
Data query method and device and electronic equipment Download PDFInfo
- Publication number
- CN112069175A CN112069175A CN202010866779.0A CN202010866779A CN112069175A CN 112069175 A CN112069175 A CN 112069175A CN 202010866779 A CN202010866779 A CN 202010866779A CN 112069175 A CN112069175 A CN 112069175A
- Authority
- CN
- China
- Prior art keywords
- query
- list
- condition
- index
- chain length
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000004590 computer program Methods 0.000 claims description 10
- 238000004891 communication Methods 0.000 description 5
- 230000004069 differentiation Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
Abstract
The embodiment of the invention provides a data query method, a data query device and electronic equipment. The method comprises the following steps: and presetting an index of a list with each chain length being larger than a preset threshold value in the database. Receiving a query request input by a user; the query request carries a query condition and a target identifier. Determining a query list in the database according to the target identification; the query list is a list corresponding to the user identifier matched with the target identifier. And under the condition that the chain length of the query list is greater than a preset threshold value and the query condition meets a first preset condition, querying in the query list in an index mode to obtain a query result. According to the embodiment of the invention, indexes do not need to be set for each list in the database, so that the storage data volume of the indexes is reduced, and the storage space is saved. And the list with the chain length larger than the preset threshold value is queried in an indexing mode, so that the query time can be greatly shortened, and the query performance is improved.
Description
Technical Field
The present invention relates to the field of data communications, and in particular, to a method and an apparatus for querying data, and an electronic device.
Background
With the continuous development of internet technology, the amount of information on the network is larger and larger. Thus, a large amount of data needs to be stored in the database. Databases are typically built using Structured Query Language (SQL) such as MySql or some other common SQL product.
In order to mainly meet the requirements of reading, writing, filtering, sequencing and the like under multiple scenes, a database can be constructed in a distributed key-list storage mode to store data. Wherein key is the identifier, list is the list of the corresponding identifier. For example, when the key is the user identifier, the list is all information about the user indicated by the user identifier. The storage form can provide stronger performance, better stability, larger storage capacity and lower operation and maintenance cost.
However, as the service data is increased, the chain length of list is also increased. When querying the data in the list, the list needs to be scanned in a full table. If the chain length of list is too large, the scanning time will be long. Therefore, a cutoff threshold is set for each list, and only a cutoff threshold amount of data is scanned when the chain length of the list exceeds the cutoff threshold. For example, the chain length of the list is 2 ten thousand, the truncation threshold is 1 ten thousand, and when the list is scanned, only 1 ten thousand pieces of data of the list are scanned. However, the list scanned by the system is not complete enough, and the query result returned to the user is not accurate enough. And the truncation threshold is large, and the time consumption is also long for scanning threshold amount of data in the stage.
Disclosure of Invention
In view of the above problems, embodiments of the present invention are proposed to provide a method, an apparatus and an electronic device for data query, which overcome the above problems or at least partially solve the above problems.
In one aspect, an embodiment of the present invention provides a method for querying data, where the method includes:
presetting an index of a list with each chain length being larger than a preset threshold in a database; wherein each list corresponds to a user identifier;
receiving a query request input by a user; wherein, the query request carries query conditions and target identification;
determining a query list in the database according to the target identification; the query list is a list corresponding to the user identification matched with the target identification;
and under the condition that the chain length of the query list is greater than a preset threshold value and the query condition meets a first preset condition, querying in the query list in an index mode to obtain a query result.
Optionally, the indexing of the list in which each chain length in the preset database is greater than the preset threshold includes:
detecting the chain length of a list corresponding to each user identifier in the database; wherein the chain length of the list is directly proportional to the data size of the data stored in the list;
determining a target list according to the chain length of the list corresponding to each user identifier; the target list is a list with the chain length being larger than a preset threshold value;
an index is created for each of the target lists.
Optionally, in a case that the number of the target lists is at least two, the creating an index for each target list includes:
adding the user identification corresponding to each target list into a pre-established queue;
and concurrently executing the creation of indexes for the target lists corresponding to the user identifications in the queue through different threads.
Optionally, creating the index includes: creating a full index and creating a delta index.
Optionally, when the chain length of the query list is greater than a preset threshold and the query condition meets a first preset condition, performing query in the query list in an indexed manner, including:
determining condition information of indexes corresponding to the query list in the query conditions and the discrimination of the indexes corresponding to the condition information under the condition that the chain length of the query list is greater than a preset threshold value;
and under the condition that the discrimination of the index corresponding to the condition information is greater than or equal to the target discrimination, adding the query condition into an index queue, and executing query on the query condition in the index queue in an index mode.
Optionally, when the chain length of the query list is greater than a preset threshold and the query condition meets a second preset condition, the query list is queried in a full-table scanning manner to obtain a query result.
Optionally, when the chain length of the query list is greater than a preset threshold and the query condition meets a second preset condition, performing query in the query list in a full-table scanning manner, including:
when the chain length of the query list is greater than a preset threshold value and the query condition meets a second preset condition, adding the query condition into a slow queue, executing the query condition in the slow queue, and querying in a full-table scanning manner;
the second preset condition is that condition information of indexes corresponding to the query list does not exist in the query condition; or the condition information corresponding to the index of the query list exists, and the discrimination of the index corresponding to the condition information is smaller than the target discrimination.
On the other hand, an embodiment of the present invention further provides a device for querying data, where the device includes:
the index module is used for presetting indexes of lists with each chain length being larger than a preset threshold value in the database; wherein each list corresponds to a user identifier;
the receiving module is used for receiving a query request input by a user; wherein, the query request carries query conditions and target identification;
the determining module is used for determining a query list in the database according to the target identification; the query list is a list corresponding to the user identification matched with the target identification;
and the first query module is used for querying in the query list in an index manner to obtain a query result under the condition that the chain length of the query list is greater than a preset threshold and the query condition meets a first preset condition.
In yet another aspect, the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps in the method for querying data as described above are implemented.
In still another aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps in the method for querying data as described above.
In the embodiment of the invention, the index of the list with each chain length being greater than the preset threshold value in the preset database is used; the indexes are arranged on the lists with the ultra-long chains in the database, and the indexes do not need to be arranged on each list, so that the storage data volume of the indexes is reduced, and the storage space is saved. Wherein each list corresponds to a user identifier. Receiving a query request input by a user; the query request carries a query condition and a target identifier. Determining a query list in the database according to the target identification; the query list is a list corresponding to the user identifier matched with the target identifier. And under the condition that the chain length of the query list is greater than a preset threshold value and the query condition meets a first preset condition, querying in the query list in an index mode to obtain a query result. According to the embodiment of the invention, the query is carried out in the list with the chain length larger than the preset threshold value in an indexing mode, so that the query time can be greatly shortened, and the query performance is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
FIG. 1 is a flow chart illustrating steps of a method for querying data according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating the steps of setting an index according to an embodiment of the present invention;
FIG. 3 is a block diagram of an apparatus for querying data according to an embodiment of the present invention;
fig. 4 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
In various embodiments of the present invention, it should be understood that the sequence numbers of the following processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
Referring to fig. 1, an embodiment of the present invention provides a method for querying data, where the method includes:
It should be noted that the user identification is identification information for distinguishing different lists. Each list corresponds to a subscriber identity. The user identities for different lists are different. Preferably, all information of the same user is stored in the list corresponding to the user identifier. The database may store data using a data table. Preferably, the list corresponding to each user identifier is a different part of the same data table in the database.
Here, the lists with the chain length less than or equal to the preset threshold may not be provided with an index, and a last Least Used (LRU) cache is performed on the full chains of the lists, so that when data is queried in the lists, a full-table scanning mode is guaranteed to have higher performance. The preset threshold may be set by itself, for example, 2 thousands, but is not limited thereto.
The preset index may be set when the data table in the database changes, but is not limited to this. Specifically, the time for setting the index can be set by itself.
It should be noted that the query request carries a query condition and a target identifier; the query request here may be a request generated by a user through an operation of inputting, clicking, and the like. The target identifier corresponds to a user identifier in the database, for example, when the database indicates different accounts through different user identifiers, that is, each user identifier corresponds to an account. The target identification is a user identification corresponding to the current login account. The query condition may be a keyword in the content input by the user, but is not limited thereto.
In this step, the query list is a list corresponding to the user identifier matched with the target identifier. Preferably, a unique list can be determined in the database by each target identifier, and the user identifier corresponding to the list is matched with the target identifier. Here, matching may be the same, but is not limited thereto.
And 104, under the condition that the chain length of the query list is greater than a preset threshold value and the query condition meets a first preset condition, querying in the query list in an index mode to obtain a query result.
It should be noted that the first preset condition may be set by itself. Specifically, the first preset condition may be, but is not limited to, a keyword included in the query condition and corresponding to an index in the query list. For example, the index of the query list includes age, gender; and when the query condition contains the age, querying in the query list in an index mode to obtain a query result.
In the embodiment of the invention, the index of the list with each chain length being greater than the preset threshold value in the preset database is used; the indexes are arranged on the lists with the ultra-long chains in the database, and the indexes do not need to be arranged on each list, so that the storage data volume of the indexes is reduced, and the storage space is saved. Wherein each list corresponds to a user identifier. Receiving a query request input by a user; the query request carries a query condition and a target identifier. Determining a query list in a database according to the target identification; the query list is a list corresponding to the user identifier matched with the target identifier. And under the condition that the chain length of the query list is greater than a preset threshold value and the query condition meets a first preset condition, querying in the query list in an index mode to obtain a query result. According to the embodiment of the invention, the query is carried out in the list with the chain length larger than the preset threshold value in an indexing mode, so that the query time can be greatly shortened, and the query performance is improved.
As shown in fig. 2, the above step 101: the index of the list with each chain length greater than the preset threshold in the preset database comprises:
It should be noted that the chain length of the list is directly proportional to the amount of data stored in the list. Preferably, each piece of data in the list corresponds to one line of data, and the larger the amount of data, the larger the number of lines in the list. Thus, the chain length of the list may be the number of rows of the list.
Here, the target list is a list in which the chain length is greater than a preset threshold. Wherein the number of the target list is at least one.
It should be noted that an index needs to be created for each target list, and the index created here is used for querying data in the target list, and is not used for finding the target list.
Preferably, in the case that the number of the target lists is at least two, creating an index for each target list includes:
adding the user identification corresponding to each target list into a pre-established queue;
and through different threads, creating indexes for the target lists corresponding to the user identifications in the queue by concurrent execution.
The queue is a parallel queue, and indexes are created for a plurality of target lists simultaneously by adopting different threads by utilizing the characteristic of concurrent execution of the parallel queue. Meanwhile, the concurrency limit of the parallel queue is utilized to avoid excessive threads for creating indexes at the same time, so that too many Input/Output (IO) resources are avoided being consumed, and the stability of service is influenced. Here, when creating the index, the creation of the index is performed using an asynchronous task. By combining asynchronous tasks and parallel queues, the method ensures that each thread creates an index of a target list.
Preferably, creating the index comprises: creating a full index and creating a delta index. The creating of the full index is full creating, namely creating a snapshot, scanning all data of the target list, and creating the index. The creation of the increment index is increment creation, namely in the process of creating the whole cache, the increment request for writing and changing the target list is replayed, and the index is created for changing and writing.
In the embodiment of the invention, before the query request is received, the index of the list is set, so that the index of the list can be used for querying when the query is carried out according to the query request. But not every time a query request is received it is necessary to set the index of the list first.
On the basis of the foregoing embodiments of the present invention, in the embodiments of the present invention, when the chain length of the query list is greater than a preset threshold and the query condition meets a first preset condition, querying in the query list in an index manner includes:
determining condition information of indexes corresponding to the query list in the query conditions and the discrimination of the indexes corresponding to the condition information under the condition that the chain length of the query list is greater than a preset threshold value;
and under the condition that the discrimination of the index corresponding to the condition information is greater than or equal to the target discrimination, adding the query condition into the index queue, and executing the query condition in the index queue in a concurrent manner by adopting an index mode.
It should be noted that the degree of distinction of the index may be set in advance, and a different degree of distinction may be set for each index item. For example, the index includes: time index entry, age index entry, gender index entry. The time index item can include a plurality of different time instants, and when the data is inquired through time, the data can be narrowed to a smaller range, so that the data can be set to have higher discrimination. The age index may also include a plurality of different ages, and when data is queried by age, the data may be narrowed to a smaller range, so that it may be set to have a higher degree of distinction. The gender index usually only includes male and female gender, and when data is queried according to gender, the data cannot be narrowed to a smaller range, so that a lower degree of distinction can be set. The target discrimination can be set according to the discrimination of different index items. For example, the time index has a differentiation degree of 8, the age index has a differentiation degree of 6, the gender index has a differentiation degree of 2, and the target differentiation degree may be 5, but is not limited thereto.
The index queue is a parallel queue. When there are multiple query requests, there may be multiple elements in the index queue, each element being a query condition for a query request. And simultaneously responding to a plurality of query requests by adopting different threads by utilizing the characteristic of concurrent execution of the parallel queues. Here, when a query is made, the query is executed using an asynchronous task. The query conditions in the index queue are obtained through asynchronous tasks, and then query is carried out in a corresponding target list in an index mode according to the query conditions. By combining asynchronous tasks and parallel queues, each thread is guaranteed to execute a query task.
According to the embodiment of the invention, under the condition that a plurality of query requests exist, the query operation is executed concurrently through the index queue, and the plurality of query requests are responded at the same time. The time for the user to wait for the query results can be shortened.
On the basis of the embodiments of the present invention, in the embodiments of the present invention, when the chain length of the query list is greater than the preset threshold and the query condition meets the second preset condition, the query is performed in the query list in a full-table scanning manner, so as to obtain a query result.
It should be noted that, when the chain length of the query list is greater than a preset threshold, it is determined which query mode is to be used for querying in the query list according to the query condition, the first preset condition, and the second preset condition. If the query condition meets a first preset condition, selecting an index mode by the query mode; and if the query condition meets a second preset condition, selecting a full-table scanning mode by the query mode. The second preset condition can be set by itself, but the second preset condition is not in conflict with the first preset condition, and the condition that the query condition meets both the first preset condition and the second preset condition does not exist.
Specifically, when the chain length of the query list is greater than a preset threshold and the query condition meets a second preset condition, the query is performed in the query list in a full-table scanning manner, including:
when the chain length of the query list is greater than a preset threshold value and the query condition meets a second preset condition, adding the query condition into the slow queue, executing the query condition in the slow queue, and querying in a full-table scanning mode;
the second preset condition is that condition information of indexes corresponding to the query list does not exist in the query conditions; or the condition information corresponding to the index of the query list exists, and the distinguishing degree of the index corresponding to the condition information is smaller than the target distinguishing degree.
If the query condition meets the second preset condition, it is indicated that the query cannot be performed in an index manner, or the query performed in the index manner does not have a good query effect.
The slow queue is a parallel queue. When there are multiple query requests, there may be multiple elements in the slow queue, each element being a query condition for a query request. And simultaneously responding to a plurality of query requests by adopting different threads by utilizing the characteristic of concurrent execution of the parallel queues. Here, when a query is made, the query is executed using an asynchronous task. The query conditions in the slow queue are obtained through the asynchronous task, and then query is carried out in a full-table scanning mode in the corresponding target list according to the query conditions. By combining asynchronous tasks and parallel queues, each thread is guaranteed to execute a query task. The slow queue and the index queue are two different queues, and each queue has respective memory use limit and thread use limit, so that the query request corresponding to the slow queue cannot influence the query request corresponding to the index queue. Meanwhile, the slow queue limits available resources of each query request, and even if a certain query request is a slow request, other query requests cannot be influenced.
On the basis of the above embodiments of the present invention, in the embodiments of the present invention, when the chain length of the query list is less than or equal to the preset threshold, the query list is queried in a full-table scanning manner.
It should be noted that, in the case that the chain length of the query list is less than or equal to the preset threshold, the query is performed in the query list in a full-table scanning manner, including:
and under the condition that the chain length of the query list is less than or equal to a preset threshold value, adding the query conditions into the common queue, concurrently executing the query conditions in the common queue, and querying in a full-table scanning mode.
The normal queue is a parallel queue. When there are multiple query requests, there may be multiple elements in the normal queue, each element being a query condition for a query request. And simultaneously responding to a plurality of query requests by adopting different threads by utilizing the characteristic of concurrent execution of the parallel queues. Here, when a query is made, the query is executed using an asynchronous task. The query conditions in the common queue are obtained through the asynchronous task, and then query is carried out in a full-table scanning mode in the corresponding target list according to the query conditions. By combining asynchronous tasks and parallel queues, each thread is guaranteed to execute a query task. The normal queue, the slow queue and the index queue are different queues, and each queue has respective memory use limit and thread use limit.
Referring to fig. 3, an embodiment of the present invention further provides an apparatus for querying data, where the apparatus includes:
the index module 31 is configured to preset an index of a list in the database, where each chain length is greater than a preset threshold; wherein each list corresponds to a user identifier;
a receiving module 32, configured to receive a query request input by a user; the query request carries a query condition and a target identifier;
a determining module 33, configured to determine a query list in the database according to the target identifier; the query list is a list corresponding to the user identification matched with the target identification;
the first query module 34 is configured to query the query list in an index manner to obtain a query result when the chain length of the query list is greater than a preset threshold and the query condition meets a first preset condition.
Optionally, the indexing module 31 includes:
the detection unit is used for detecting the chain length of the list corresponding to each user identifier in the database; wherein the chain length of the list is in direct proportion to the data size of the data stored in the list;
the determining unit is used for determining a target list according to the chain length of the list corresponding to each user identifier; the target list is a list with the chain length being larger than a preset threshold value;
and the creating unit is used for creating an index for each target list.
Optionally, in the case that the number of the target lists is at least two, the creating unit is specifically configured to add the user identifier corresponding to each target list into a pre-created queue; and through different threads, creating indexes for the target lists corresponding to the user identifications in the queue by concurrent execution.
Optionally, creating the index includes: creating a full index and creating a delta index.
Optionally, the first query module 34 includes:
the first query unit is used for determining condition information of indexes corresponding to the query list in the query conditions and the discrimination of the indexes corresponding to the condition information under the condition that the chain length of the query list is greater than a preset threshold value;
and the second query unit is used for adding the query conditions into the index queue and concurrently executing query conditions in the index queue in an index mode under the condition that the discrimination of the index corresponding to the condition information is greater than or equal to the target discrimination.
Optionally, the apparatus further comprises: and the second query module is used for querying in the query list in a full-table scanning manner under the condition that the chain length of the query list is greater than the preset threshold and the query condition meets a second preset condition to obtain a query result.
Optionally, the second query module is specifically configured to add the query condition to the slow queue and concurrently execute the query condition in the slow queue to query in a full-table scanning manner when the chain length of the query list is greater than a preset threshold and the query condition meets a second preset condition;
the second preset condition is that condition information of indexes corresponding to the query list does not exist in the query conditions; or the condition information corresponding to the index of the query list exists, and the distinguishing degree of the index corresponding to the condition information is smaller than the target distinguishing degree.
The data query device provided by the embodiment of the present invention can implement each process implemented by the data query method in the embodiments of the methods in fig. 1 and fig. 2, and is not described herein again to avoid repetition.
In the embodiment of the invention, the index module is used for presetting the index of a list with each chain length being larger than a preset threshold value in a database; wherein each list corresponds to a user identifier. The indexes are arranged on the lists with the ultra-long chains in the database, and the indexes do not need to be arranged on each list, so that the storage data volume of the indexes is reduced, and the storage space is saved. The receiving module is used for receiving a query request input by a user; the query request carries a query condition and a target identifier; the determining module is used for determining a query list in the database according to the target identification; the query list is a list corresponding to the user identification matched with the target identification; and the first query module is used for querying in the query list in an index manner to obtain a query result under the condition that the chain length of the query list is greater than a preset threshold and the query condition meets the first preset condition. According to the embodiment of the invention, the query is carried out in the list with the chain length larger than the preset threshold value in an indexing mode, so that the query time can be greatly shortened, and the query performance is improved.
In another aspect, an embodiment of the present invention further provides an electronic device, which includes a memory, a processor, a bus, and a computer program stored in the memory and executable on the processor, where the processor implements the steps in the above data query method when executing the program.
For example, fig. 4 shows a schematic physical structure diagram of an electronic device.
As shown in fig. 4, the electronic device may include: a processor (processor)410, a communication Interface 420, a memory (memory)430 and a communication bus 440, wherein the processor 410, the communication Interface 420 and the memory 430 are communicated with each other via the communication bus 440. The processor 410 may call logic instructions in the memory 430 to perform the following method:
presetting an index of a list with each chain length being larger than a preset threshold in a database; wherein each list corresponds to a user identifier;
receiving a query request input by a user; wherein, the query request carries query conditions and target identification;
determining a query list in the database according to the target identification; the query list is a list corresponding to the user identification matched with the target identification;
and under the condition that the chain length of the query list is greater than a preset threshold value and the query condition meets a first preset condition, querying in the query list in an index mode to obtain a query result.
In addition, the logic instructions in the memory 430 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In still another aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is implemented to perform the method for querying data provided in the foregoing embodiments when executed by a processor, for example, the method includes:
presetting an index of a list with each chain length being larger than a preset threshold in a database; wherein each list corresponds to a user identifier;
receiving a query request input by a user; wherein, the query request carries query conditions and target identification;
determining a query list in the database according to the target identification; the query list is a list corresponding to the user identification matched with the target identification;
and under the condition that the chain length of the query list is greater than a preset threshold value and the query condition meets a first preset condition, querying in the query list in an index mode to obtain a query result.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method of data querying, the method comprising:
presetting an index of a list with each chain length being larger than a preset threshold in a database; wherein each list corresponds to a user identifier;
receiving a query request input by a user; wherein, the query request carries query conditions and target identification;
determining a query list in the database according to the target identification; the query list is a list corresponding to the user identification matched with the target identification;
and under the condition that the chain length of the query list is greater than a preset threshold value and the query condition meets a first preset condition, querying in the query list in an index mode to obtain a query result.
2. The method of claim 1, wherein the step of presetting the index of the list with each chain length greater than the preset threshold in the database comprises:
detecting the chain length of a list corresponding to each user identifier in the database; wherein the chain length of the list is directly proportional to the data size of the data stored in the list;
determining a target list according to the chain length of the list corresponding to each user identifier; the target list is a list with the chain length being larger than a preset threshold value;
an index is created for each of the target lists.
3. The method of claim 2, wherein in the case that the number of the target lists is at least two, the creating an index for each target list comprises:
adding the user identification corresponding to each target list into a pre-established queue;
and concurrently executing the creation of indexes for the target lists corresponding to the user identifications in the queue through different threads.
4. The method of claim 2 or 3, wherein creating an index comprises: creating a full index and creating a delta index.
5. The method according to claim 1, wherein in a case that the chain length of the query list is greater than a preset threshold and the query condition meets a first preset condition, performing query in the query list in an indexed manner includes:
determining condition information of indexes corresponding to the query list in the query conditions and the discrimination of the indexes corresponding to the condition information under the condition that the chain length of the query list is greater than a preset threshold value;
and under the condition that the discrimination of the index corresponding to the condition information is greater than or equal to the target discrimination, adding the query condition into an index queue, and executing query on the query condition in the index queue in an index mode.
6. The method according to claim 1, wherein in a case that the chain length of the query list is greater than a preset threshold and the query condition meets a second preset condition, the query list is queried in a full-table scanning manner to obtain a query result.
7. The method according to claim 6, wherein in a case that the chain length of the query list is greater than a preset threshold and the query condition meets a second preset condition, performing query in the query list in a full-table scanning manner includes:
when the chain length of the query list is greater than a preset threshold value and the query condition meets a second preset condition, adding the query condition into a slow queue, executing the query condition in the slow queue, and querying in a full-table scanning manner;
the second preset condition is that condition information of indexes corresponding to the query list does not exist in the query condition; or the condition information corresponding to the index of the query list exists, and the discrimination of the index corresponding to the condition information is smaller than the target discrimination.
8. An apparatus for data query, the apparatus comprising:
the index module is used for presetting indexes of lists with each chain length being larger than a preset threshold value in the database; wherein each list corresponds to a user identifier;
the receiving module is used for receiving a query request input by a user; wherein, the query request carries query conditions and target identification;
the determining module is used for determining a query list in the database according to the target identification; the query list is a list corresponding to the user identification matched with the target identification;
and the first query module is used for querying in the query list in an index manner to obtain a query result under the condition that the chain length of the query list is greater than a preset threshold and the query condition meets a first preset condition.
9. An electronic device comprising a processor, a memory and a computer program stored on the memory and executable on the processor, characterized in that the computer program, when executed by the processor, implements the steps of the method of data querying according to any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of data querying according to one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010866779.0A CN112069175B (en) | 2020-08-25 | 2020-08-25 | Data query method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010866779.0A CN112069175B (en) | 2020-08-25 | 2020-08-25 | Data query method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112069175A true CN112069175A (en) | 2020-12-11 |
| CN112069175B CN112069175B (en) | 2024-03-29 |
Family
ID=73660679
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010866779.0A Active CN112069175B (en) | 2020-08-25 | 2020-08-25 | Data query method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112069175B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112733001A (en) * | 2021-01-14 | 2021-04-30 | 上海蓝云网络科技有限公司 | Method and device for acquiring subscription information and electronic equipment |
| CN113064901A (en) * | 2021-04-06 | 2021-07-02 | 北京瑞卓喜投科技发展有限公司 | Method and device for forming data micro-index in contract on chain and electronic equipment |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060294058A1 (en) * | 2005-06-28 | 2006-12-28 | Microsoft Corporation | System and method for an asynchronous queue in a database management system |
| US20110119249A1 (en) * | 2009-11-13 | 2011-05-19 | Oracle International Corporation | Index backbone join |
| CN103902701A (en) * | 2014-03-31 | 2014-07-02 | 北京车商汇软件有限公司 | Data storage system and data storage method |
| US20150220584A1 (en) * | 2014-02-03 | 2015-08-06 | Codefutures Corporation | Dynamic modification of a database data structure |
| US20150331910A1 (en) * | 2014-04-28 | 2015-11-19 | Venkatachary Srinivasan | Methods and systems of query engines and secondary indexes implemented in a distributed database |
| WO2015195830A2 (en) * | 2014-06-18 | 2015-12-23 | Alibaba Group Holding Limited | Data query method and apparatus |
| CN105205071A (en) * | 2014-06-24 | 2015-12-30 | 阿里巴巴集团控股有限公司 | Method and system for showing data table |
| CN105224609A (en) * | 2015-09-07 | 2016-01-06 | 北京金山安全软件有限公司 | Index query method and device |
| CN106407250A (en) * | 2015-07-28 | 2017-02-15 | 阿里巴巴集团控股有限公司 | Information query method, device and system, server and client side |
| CN108694188A (en) * | 2017-04-07 | 2018-10-23 | 腾讯科技（深圳）有限公司 | A kind of newer method of index data and relevant apparatus |
| CN110019192A (en) * | 2017-09-21 | 2019-07-16 | 阿里巴巴集团控股有限公司 | The search method and device of database |
| CN110162522A (en) * | 2019-05-22 | 2019-08-23 | 武汉市公安局 | A kind of distributed data search system and method |
| CN110347722A (en) * | 2019-07-11 | 2019-10-18 | 软通智慧科技有限公司 | Data capture method, device, equipment and storage medium based on HBase |
| CN110515990A (en) * | 2019-07-23 | 2019-11-29 | 华信永道(北京)科技股份有限公司 | Data query methods of exhibiting and inquiry display systems |
| CN111190897A (en) * | 2019-11-07 | 2020-05-22 | 腾讯科技（深圳）有限公司 | Information processing method, information processing apparatus, storage medium, and server |
| CN111209296A (en) * | 2019-12-31 | 2020-05-29 | 航天信息股份有限公司企业服务分公司 | Database access method and device, electronic equipment and storage medium |
-
2020
- 2020-08-25 CN CN202010866779.0A patent/CN112069175B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060294058A1 (en) * | 2005-06-28 | 2006-12-28 | Microsoft Corporation | System and method for an asynchronous queue in a database management system |
| US20110119249A1 (en) * | 2009-11-13 | 2011-05-19 | Oracle International Corporation | Index backbone join |
| US20150220584A1 (en) * | 2014-02-03 | 2015-08-06 | Codefutures Corporation | Dynamic modification of a database data structure |
| CN103902701A (en) * | 2014-03-31 | 2014-07-02 | 北京车商汇软件有限公司 | Data storage system and data storage method |
| US20150331910A1 (en) * | 2014-04-28 | 2015-11-19 | Venkatachary Srinivasan | Methods and systems of query engines and secondary indexes implemented in a distributed database |
| WO2015195830A2 (en) * | 2014-06-18 | 2015-12-23 | Alibaba Group Holding Limited | Data query method and apparatus |
| CN105205071A (en) * | 2014-06-24 | 2015-12-30 | 阿里巴巴集团控股有限公司 | Method and system for showing data table |
| CN106407250A (en) * | 2015-07-28 | 2017-02-15 | 阿里巴巴集团控股有限公司 | Information query method, device and system, server and client side |
| CN105224609A (en) * | 2015-09-07 | 2016-01-06 | 北京金山安全软件有限公司 | Index query method and device |
| CN108694188A (en) * | 2017-04-07 | 2018-10-23 | 腾讯科技（深圳）有限公司 | A kind of newer method of index data and relevant apparatus |
| CN110019192A (en) * | 2017-09-21 | 2019-07-16 | 阿里巴巴集团控股有限公司 | The search method and device of database |
| CN110162522A (en) * | 2019-05-22 | 2019-08-23 | 武汉市公安局 | A kind of distributed data search system and method |
| CN110347722A (en) * | 2019-07-11 | 2019-10-18 | 软通智慧科技有限公司 | Data capture method, device, equipment and storage medium based on HBase |
| CN110515990A (en) * | 2019-07-23 | 2019-11-29 | 华信永道(北京)科技股份有限公司 | Data query methods of exhibiting and inquiry display systems |
| CN111190897A (en) * | 2019-11-07 | 2020-05-22 | 腾讯科技（深圳）有限公司 | Information processing method, information processing apparatus, storage medium, and server |
| CN111209296A (en) * | 2019-12-31 | 2020-05-29 | 航天信息股份有限公司企业服务分公司 | Database access method and device, electronic equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| CHANG_HF: "mysql索引与全表扫描", pages 1 - 3, Retrieved from the Internet <URL:https://blog.csdn.net/chang_li/article/details/86506387> * |
| TTXS_HC: "数据库建立索引常用的规则", pages 1 - 3, Retrieved from the Internet <URL:https://blog.csdn.net/hc_ttxs/article/details/98090980> * |
| 岳彩云等: "基于ORACLE数据库的SQL性能优化", 《电脑知识与技术》, vol. 16, no. 10 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112733001A (en) * | 2021-01-14 | 2021-04-30 | 上海蓝云网络科技有限公司 | Method and device for acquiring subscription information and electronic equipment |
| CN112733001B (en) * | 2021-01-14 | 2023-09-29 | 上海蓝云网络科技有限公司 | Method and device for acquiring subscription information and electronic equipment |
| CN113064901A (en) * | 2021-04-06 | 2021-07-02 | 北京瑞卓喜投科技发展有限公司 | Method and device for forming data micro-index in contract on chain and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112069175B (en) | 2024-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108804641B (en) | Text similarity calculation method, device, equipment and storage medium | |
| US9575984B2 (en) | Similarity analysis method, apparatus, and system | |
| CN109933617B (en) | Data processing method, data processing device, related equipment and related medium | |
| CN112035599B (en) | Query method and device based on vertical search, computer equipment and storage medium | |
| CN108228799B (en) | Object index information storage method and device | |
| CN109669980B (en) | Cross-database access method and device for data | |
| CN108536739B (en) | Metadata sensitive information field identification method, device, equipment and storage medium | |
| CN109669925B (en) | Management method and device of unstructured data | |
| CN110046298A (en) | Query word recommendation method and device, terminal device and computer readable medium | |
| CN112069175A (en) | Data query method and device and electronic equipment | |
| US20190370392A1 (en) | System and method for multi-language search | |
| CN112364014A (en) | Data query method, device, server and storage medium | |
| CN110968802A (en) | User characteristic analysis method, analysis device and readable storage medium | |
| CN112434056A (en) | Method and device for inquiring detailed data | |
| CN106446080B (en) | Data query method, query service equipment, client equipment and data system | |
| CN110929207B (en) | Data processing method, device and computer readable storage medium | |
| CN116263770A (en) | Method, device, terminal equipment and medium for storing business data based on database | |
| CN110471586B (en) | Project recommendation method, apparatus, computer device and storage medium | |
| CN113849520A (en) | Intelligent identification method and device of abnormal SQL (structured query language), electronic equipment and storage medium | |
| CN111858609A (en) | Fuzzy query method and device for block chain | |
| CN109783052B (en) | Data sorting method, device, server and computer readable storage medium | |
| CN113778904B (en) | Multi-terminal system testing method and device based on integrated test case | |
| CN114943040A (en) | Object sorting method, device, equipment, medium and product | |
| CN115185984A (en) | Hbase database query node optimization method, device, equipment and storage medium | |
| CN115757948A (en) | Information pushing method and device, storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |