CN113590623A - Method, device and equipment for data deep paging query - Google Patents
Method, device and equipment for data deep paging query Download PDFInfo
- Publication number
- CN113590623A CN113590623A CN202110856921.8A CN202110856921A CN113590623A CN 113590623 A CN113590623 A CN 113590623A CN 202110856921 A CN202110856921 A CN 202110856921A CN 113590623 A CN113590623 A CN 113590623A
- Authority
- CN
- China
- Prior art keywords
- page number
- query
- data
- target page
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000012216 screening Methods 0.000 claims abstract description 41
- 230000015654 memory Effects 0.000 claims abstract description 24
- 230000001174 ascending effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 230000033772 system development Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- 230000003936 working memory Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a technical scheme for data deep paging query, which is characterized in that total data volume meeting screening conditions and a total page number corresponding to the total data volume are determined based on an obtained data query instruction; acquiring a target page number and a maximum allowable query page number; if the target page number is larger than the maximum allowable query page number, performing data query based on a screening condition, the maximum allowable query page number and an initial index, and determining the last data record in the queried data volume; taking the next data record of the last data record as a new initial index, determining the difference between the target page number and the maximum allowable page number as a new target page number, and repeating the steps until the target page number is less than or equal to the maximum allowable query page number; and performing data query based on the screening condition, the target page number and the initial index to obtain a query result. The method can ensure that the query performance is not reduced, does not need to occupy additional memory resources, and can reflect the change of data in the query result in real time.
Description
Technical Field
The application relates to the technical field of computer data processing, in particular to a technology for deep paging query of data.
Background
Data paging queries are a common usage scenario in database system development or applications. When the number of pages to be queried is too large, that is, deep paging is queried (without explicit definition, which is different from system to system), the database system may be greatly affected, for example, the resource overhead such as memory becomes large, the performance such as query speed is reduced, and even the database system may be crushed, so that the database system is unusable.
In order to avoid the above situation as much as possible, the conventional database system usually makes some page number limits on the conventional data paging query, and the data paging query exceeding the page number limits adopts a scroll (snapshot cursor) mode and the like to realize the data deep paging query. However, a snapshot is built on a large amount of data by adopting a scroll to realize deep paging query of the data, more memory resources are still occupied, and the change of the data in the database cannot be reflected in a deep paging query result in real time.
Disclosure of Invention
The application aims to provide a technical scheme for data deep paging query, which is used for solving the technical problems that the existing data deep paging query method additionally occupies memory resources and the change of data cannot be reflected in a query result in real time.
According to one aspect of the present application, a method for data deep paging query is provided, wherein the method comprises:
determining total data volume meeting preset screening conditions based on the obtained data query instruction, and determining total page numbers corresponding to the total data volume based on preset paging parameters;
acquiring a target page number, and determining a maximum allowable query page number based on a preset deep paging minimum query data volume and the preset paging parameter, wherein the target page number is less than or equal to the total page number;
if the target page number is larger than the maximum allowable query page number, performing data query based on the preset screening condition, the maximum allowable query page number and an initial index, and determining the last data record in the queried data volume, wherein the initial index of the initial query is 1;
taking the next data record of the last data record as a new initial index, determining the difference between the target page number and the maximum allowable page number as a new target page number, and repeating the previous step and the current step until the target page number is less than or equal to the maximum allowable page number;
and performing data query based on the preset screening condition, the target page number and the initial index to obtain a query result.
Optionally, wherein the taking the next data record of the last data record as the new start index includes:
and determining the next data record in the same direction as the sorting field in the last data record as a new initial index based on the preset sorting condition.
Optionally, wherein the taking the next data record of the last data record as the new start index includes:
and determining the next data record in the same direction as the sorting field and the main key in the last data record as a new initial index based on the preset sorting condition and the preset main key sorting direction.
Optionally, the method for data deep paging query further includes:
and returning the total data volume and the query result.
According to another aspect of the present application, there is also provided an apparatus for deep paging query of data, wherein the apparatus includes:
the device comprises a first module, a second module and a third module, wherein the first module is used for determining total data volume meeting preset screening conditions based on an obtained data query instruction, and determining total page numbers corresponding to the total data volume based on preset paging parameters;
a second module, configured to obtain a target page number, and determine a maximum allowable query page number based on a preset deep paging minimum query data amount and the preset paging parameter, where the target page number is less than or equal to the total page number;
a third module, configured to query data based on the preset screening condition, the maximum allowed query page number, and an initial index if the target page number is greater than the maximum allowed query page number, and determine a last data record in the queried data volume, where the initial index of the initial query is 1;
a fourth module, configured to use a next data record of the last data record as a new start index, determine a difference between the target page number and the maximum allowable page number as a new target page number, and repeat operations of the third module and the present module until the target page number is less than or equal to the maximum allowable query page number;
and the fifth module is used for carrying out data query based on the preset screening condition, the target page number and the initial index to obtain a query result.
Compared with the prior art, the technical scheme for data deep paging query is that firstly, the total data size meeting the preset screening condition is determined based on the obtained data query instruction, and the total page number corresponding to the total data size is determined based on the preset paging parameter; then, acquiring a target page number input by a user, and determining a maximum allowable query page number based on a preset deep paging minimum query data volume and the preset paging parameter, wherein the target page number is less than or equal to the total page number; if the target page number is larger than the maximum allowable query page number, performing data query based on the preset screening condition, the maximum allowable query page number and an initial index, and determining the last data record in the queried data volume, wherein the initial index of the initial query is 1; then taking the next data record of the last data record as a new initial index, determining the difference between the target page number and the maximum allowable page number as a new target page number, and repeating the operations of a third module and the module until the target page number is less than or equal to the maximum allowable query page number; and finally, performing data query based on the preset screening condition, the target page number and the initial index to obtain a query result. The method can realize data deep paging query, the data scanning amount and the occupied memory resource of each query are the same as those of the conventional paging query, no extra memory resource is occupied, the query performance such as query speed is not reduced, and the change of the data can be reflected in the query result in real time.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings:
FIG. 1 illustrates a flow diagram of a method for deep paging queries of data, in accordance with an aspect of the subject application;
FIG. 2 is a schematic diagram illustrating a deep paging query flow of data for a target page according to an embodiment of the present application
FIG. 3 illustrates an apparatus diagram for deep paging queries of data in accordance with another aspect of the subject application;
the same or similar reference numbers in the drawings identify the same or similar elements.
Detailed Description
The present invention is described in further detail below with reference to the attached drawing figures.
In a typical configuration of the present application, each module of the device, apparatus, and/or associated trusted party includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
In order to further explain the technical means and effects adopted by the present application, the following description clearly and completely describes the technical solution of the present application with reference to the accompanying drawings and preferred embodiments.
FIG. 1 illustrates a flow diagram of a method for deep paging queries for data in one aspect of the application, wherein the method of an embodiment comprises:
s11, determining total data size meeting preset screening conditions based on the obtained data query instruction, and determining total page numbers corresponding to the total data size based on preset paging parameters;
s12, acquiring a target page number, and determining a maximum allowable query page number based on a preset deep paging minimum query data volume and the preset paging parameter, wherein the target page number is less than or equal to the total page number;
s13, if the target page number is larger than the maximum allowable query page number, performing data query based on the preset screening condition, the maximum allowable query page number and the initial index, and determining the last data record in the queried data volume, wherein the initial index of the initial query is 1;
s14, taking the next data record of the last data record as a new initial index, determining the difference between the target page number and the maximum allowable page number as a new target page number, and repeating the previous step and the current step until the target page number is less than or equal to the maximum allowable page number;
s15, based on the preset screening condition, the target page number and the initial index, data is queried to obtain a query result.
In the present application, the method is performed by an apparatus 100, the apparatus 100 is a computer device and/or a cloud, the computer device includes but is not limited to a personal computer, a notebook computer, an industrial computer, a network host, a single network server, a plurality of network server sets; the Cloud is made up of a large number of computers or web servers based on Cloud Computing (Cloud Computing), which is a type of distributed Computing, a virtual supercomputer consisting of a collection of loosely coupled computers.
The computer device and/or cloud are merely examples, and other existing or future devices and/or resource sharing platforms, as applicable to the present application, are also intended to be included within the scope of the present application and are hereby incorporated by reference.
Wherein the device 100 is a database server and/or a search engine server, such as: mysql database server, Elasticsearch distributed search engine server, etc.
In the step S11, when the device 100 receives a data query instruction sent by a user through a client, the total data size meeting the preset screening condition is determined according to the preset screening condition in the instruction, and the total page number corresponding to the total data size is determined based on the preset paging parameter.
The preset screening condition is determined according to a specific user data query requirement and is related to a specific application scenario, for example, a query instruction for a mysql database:
select from table name, name id >5and name ═ and' order by name limit start, pagesize
Where id >5and name ═ and' are the screening conditions.
The preset paging parameter may be a default value set by a system in the database server and/or the search engine server, or may be an external parameter brought by a data query instruction. For example, the pagesize in the query instruction to the mysql database described above is the externally substituted paging parameter.
Wherein, the device 100 obtains the total data volume meeting the preset screening condition from the database server and/or the search engine server according to the preset screening condition in the obtained data query instruction, and determines the total page number corresponding to the total data volume according to the preset paging parameter in the obtained query instruction, wherein,
total page number-total data size/paging parameter
In this embodiment, in step S12, the apparatus 100 obtains a target page number, and determines a maximum allowable query page number based on a preset deep paging minimum query data size and the preset paging parameter, where the target page number is less than or equal to the total page number. The preset deep paging minimum query data size is determined by the device 100, and is usually set as the maximum query data size that does not affect the conventional paging query performance of the device 100, for example, when the device 100 performs paging query on data within 10000 data records, the query performance is almost unchanged, and when the paging query is performed on data exceeding 10000 data records, the query performance is sharply reduced, and then the maximum query data size of the device 100 is determined to be 10000 data records.
Wherein the content of the first and second substances,
maximum allowed query page number as deep paging minimum query data size/paging parameter
The target page number is input by the user through the client, the target page number acquired by the device 100 should be smaller than the total page number, and if the target page number is larger than the total page number, the device 100 may feed back abnormal information to the client, such as "data is empty", "out of query range", and the like. Typically, the target page number is converted to the starting index in the data query instruction after being input by the user. For example, the startrow in the query instruction for the mysql database may be the corresponding start index after the conversion for the input target page number, where startrow is the target page number and the paging parameter +1, and if the query is for the first page data, startrow is 1.
In a preferred embodiment, after determining the total data size and the total page number according to the obtained data query instruction, the device 100 further returns a first page data result in the total data size to the client, and returns the total page number, and the user performs data paging query by inputting a target page number smaller than the total page number through the client.
In order to determine whether the data query for the target page belongs to a deep paging query, the apparatus 100 needs to determine a size between the target page and a maximum allowed query page, if the target page is less than or equal to the maximum allowed query page, the apparatus 100 determines that the data query for the target page is a normal data paging query, if not, in step S13, if the target page is greater than the maximum allowed query page, it indicates that the data query for the target page is a deep paging query, and the apparatus 100 performs the data query based on the preset screening condition, the maximum allowed query page and a start index, and determines a last data record in the queried data volume, where the start index of the initial query is 1.
In this embodiment, the deep paging query of the apparatus 100 for the target page number may be decomposed into several regular paging queries.
The first time, the conventional paging query is performed according to the preset screening condition, the maximum allowed query page number and the starting index of 1. And if the maximum allowable query page number is L, determining the first queried data as the data of the L-th page, and determining the last data record of the L-th page. Wherein a start index of 1 indicates the first data record in the total data volume. For example, in the above target page number query command for the mysql database, if the lth page data is queried, startrow is L × page parameter + 1.
Continuing in this embodiment, in step S14, the apparatus 100 uses the next data record of the last data record as a new start index, determines the difference between the target page number and the maximum allowable page number as a new target page number, and then repeats steps S13 and this step until the target page number is less than or equal to the maximum allowable page number.
Wherein, if the initial target page number is greater than or equal to N times the maximum allowed page number, the deep paging query of the apparatus 100 for the target page number can be simplified to (N +1) times of the regular paging query. The method comprises the following steps of firstly carrying out N times of routine paging inquiry according to a preset screening condition, a maximum allowable inquiry page number and an updated initial index each time. If the target page number is M and the maximum allowable query page number is L, the data queried for the Nth time is the data of the (N x L) th page, and the last data record of the (N x L) th page is determined. After the nth conventional paging query, the latest target page number is (M-N × L), and the latest initial index is the next data record after the last data record of the (N × L) page. For example, the above target page number data query command for the mysql database, the device 100 performs N times of conventional page queries and then updates the latest starting index startrow + 1.
Optionally, in step S14, the step of taking the next data record of the last data record as a new start index includes:
and determining the next data record in the same direction as the sorting field in the last data record as a new initial index based on the preset sorting condition.
If the maximum allowable query page number is L, and the last data record determined in each conventional paging query is the last data record in the lth page of the time, based on a preset sorting condition in the data query instruction, the value of the sorting field of the last data record in the lth page of the time can be determined to identify the last data record in the queried data volume, and the next data record in the same direction as the sorting field of the last data record in the lth page of the time is determined as a new starting index. The sorting field is determined by the preset sorting condition, wherein the preset sorting condition can be a default value set by a database or a search engine system, or can be brought in by a data query instruction input by a user through a client. The sorting fields in the same direction refer to that the data records are sequentially arranged in an ascending order or a descending order direction with respect to the sorting fields in the preset sorting condition, for example, if the preset sorting condition is that the values of the sorting fields are arranged in an ascending order, the data record corresponding to the new start index is the first data record of which the value of the sorting field is greater than the value of the sorting field of the last data record of the next L-th page; if the preset sorting condition is that the sorting fields are sorted in a descending order according to the values of the sorting fields, the data record corresponding to the new initial index is the first data record of which the value of the sorting field is smaller than the value of the sorting field of the last data record of the next L-th page.
Because the sorting field in some application scenarios is not necessarily the only identifier of the data record in the acquired query data, there may be a case where the values of the sorting fields of multiple data records are all the same, if there is an error in the last data record determined by the sorting field in the preset sorting condition in each conventional paging query, it is impossible to accurately locate the last data record in each conventional paging query, and it is also possible to consider to add a primary key sort, that is, to assemble the sort of the sorting field in the preset sorting condition and the primary key sort, to accurately locate the last data record in each conventional paging query, thereby accurately determining the start index of the next conventional paging query.
Optionally, in step S14, the step of taking the next data record of the last data record as a new start index includes:
and determining the next data record in the same direction as the sorting field and the main key in the last data record as a new initial index based on the preset sorting condition and the preset main key sorting direction.
If the maximum allowable query page number is L, and the last data record determined in each conventional paging query is the last data record in the next L-th page, the value of the sorting field and the value of the main key of the last data record in the next L-th page may be determined based on the preset sorting condition and the preset main key sorting direction in the data query instruction, so as to accurately identify the last data record in the queried data volume, and further, the next data record in the same direction as the sorting field and the main key sorting direction of the last data record in the next L-th page may be accurately determined as a new start index. The sorting field is determined by the preset sorting condition, wherein the preset sorting condition and the preset primary key sorting direction can be default values set by a database or a search engine system, or can be brought by a data query instruction input by a user through a client. The sorting fields and the main keys are all arranged in the same direction, namely the data records are arranged in the ascending or descending direction of the sorting fields in the preset sorting condition and in the ascending or descending direction of the preset main keys, for example, if the preset sorting condition is that the values of the sorting fields are arranged in the ascending order and the sorting direction of the preset main keys is that the sorting direction is the descending direction of the main keys, the data record corresponding to the new initial index is the first data record of which the value of the sorting field is greater than that of the sorting field of the last data record of the next L page, and if the data records with the same values of a plurality of sorting fields exist, the first data record arranged in the descending order in the sorting direction of the main keys is determined as the new initial index; if the preset sorting condition is that the data records corresponding to the new initial index are the first data records of which the values of the sorting fields are smaller than the values of the sorting fields of the last data record of the next L-th page, and if the data records with the same values of the sorting fields exist, the first data records which are arranged in the ascending order in the sorting direction of the main key are determined as the new initial index.
In this embodiment, after the device 100 completes N regular paging queries, in step S15, the device 100 performs 1 regular paging query according to the preset filtering condition, the latest target page number and the start index, and obtains the query result.
The data deep paging query process for the target page in the embodiment is illustrated in fig. 2, and the data deep paging query for the target page is decomposed into a plurality of times of conventional paging queries, so that query performance is not reduced, additional memory resources are not occupied, and the obtained query result is real-time data.
Optionally, the method for data deep paging query further includes:
s16 (not shown) returns the total data size and query result.
The device 100 also returns the total data amount determined based on the acquired data query instruction and the deep paging query result acquired based on the target page number input by the user to the client for presentation to the user.
Fig. 3 is a schematic diagram of an apparatus for deep paging query of data according to another aspect of the present application, where the apparatus includes:
a first module 31, configured to determine, based on the obtained data query instruction, a total data size that meets a preset screening condition, and determine, based on a preset paging parameter, a total page number corresponding to the total data size;
a second module 32, configured to obtain a target page number, and determine a maximum allowable query page number based on a preset deep paging minimum query data amount and the preset paging parameter, where the target page number is less than or equal to the total page number;
a third module 33, configured to perform data query based on the preset screening condition, the maximum allowed query page number, and an initial index if the target page number is greater than the maximum allowed query page number, and determine a last data record in the queried data volume, where the initial index of the initial query is 1;
a fourth module 34, configured to use a next data record of the last data record as a new start index, determine a difference between the target page number and the maximum allowable page number as a new target page number, and repeat operations of the third module and the present module until the target page number is less than or equal to the maximum allowable query page number;
a fifth module 35, configured to perform data query based on the preset screening condition, the target page number, and the initial index, to obtain a query result.
In this embodiment, the apparatus is used to implement the method of the previous embodiment, and the apparatus is integrated in the same device as the software and hardware configuration environment of the previous device 100.
The first module 31 of the device receives a data query instruction sent by a user through a client, determines total data volume meeting preset screening conditions according to preset screening conditions in the instruction, and determines total page numbers corresponding to the total data volume based on preset paging parameters. The preset screening condition is determined according to specific user data query requirements and is related to a specific application scene.
The second module 32 of the apparatus obtains a target page number, and determines a maximum allowable query page number based on a preset deep paging minimum query data size and the preset paging parameter, wherein the target page number is less than or equal to the total page number. The preset deep paging minimum query data volume is generally set as the maximum query data volume which does not influence the conventional paging query performance of the equipment by the equipment integrated with the device.
In order to determine whether the data query for the target page belongs to a deep paging query, the third module 33 of the apparatus needs to determine the size between the target page and the maximum allowed query page, and if the target page is greater than the maximum allowed query page, it indicates that the data query for the target page is a deep paging query, and then the device 100 performs the data query based on the preset screening condition, the maximum allowed query page and the start index, and determines the last data record in the queried data volume, where the start index of the first query is 1.
The fourth module 34 of the apparatus uses the next data record of the last data record determined by the third module 33 as a new start index, determines the difference between the target page number and the maximum allowable page number as a new target page number, and then repeats the operations of the third module 33 and the present module until the target page number is less than or equal to the maximum allowable page number.
A fifth module 35 of the apparatus performs 1-time conventional paging query according to a preset screening condition, the latest target page number, and the start index, and obtains a query result.
The device decomposes the data deep paging query requirement of a user aiming at the target page into a plurality of times of conventional paging queries, can ensure that the query performance is not reduced, does not need to occupy additional memory resources, and obtains the query result as real-time data.
According to yet another aspect of the present application, there is also provided a computer readable medium having stored thereon computer readable instructions executable by a processor to implement the foregoing method.
It should be noted that the present application may be implemented in software and/or a combination of software and hardware, for example, implemented using Application Specific Integrated Circuits (ASICs), general purpose computers or any other similar hardware devices. In one embodiment, the software programs referred to in the present application may be executed by a processor to implement the steps or functions described above. Also, the software programs (including associated data structures) of the present application may be stored in a computer-readable recording medium. Additionally, some of the steps or functions of the present application may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present application may be implemented as a computer program product, such as computer program instructions, which when executed by a computer, may invoke or provide methods and/or techniques in accordance with the present application through the operation of the computer. Program instructions which invoke the methods of the present application may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the present application comprises a device comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the device to perform a method and/or a solution according to the aforementioned embodiments of the present application.
According to another aspect of the present application, there is also provided an apparatus for deep paging query of data, wherein the apparatus includes:
one or more processors; and
a memory storing computer readable instructions that, when executed, cause the processor to perform operations of the method as previously described.
For example, the computer readable instructions, when executed, cause the one or more processors to: determining total data volume meeting preset screening conditions based on the obtained data query instruction, and determining total page numbers corresponding to the total data volume based on preset paging parameters; acquiring a target page number, and determining a maximum allowable query page number based on a preset deep paging minimum query data volume and the preset paging parameter, wherein the target page number is less than or equal to the total page number; if the target page number is larger than the maximum allowable query page number, performing data query based on the preset screening condition, the maximum allowable query page number and an initial index, and determining the last data record in the queried data volume, wherein the initial index of the initial query is 1; taking the next data record of the last data record as a new initial index, determining the difference between the target page number and the maximum allowable page number as a new target page number, and repeating the previous step and the current step until the target page number is less than or equal to the maximum allowable page number; and performing data query based on the preset screening condition, the target page number and the initial index to obtain a query result.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software and/or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (7)
1. A method for deep paging queries for data, the method comprising:
determining total data volume meeting preset screening conditions based on the obtained data query instruction, and determining total page numbers corresponding to the total data volume based on preset paging parameters;
acquiring a target page number, and determining a maximum allowable query page number based on a preset deep paging minimum query data volume and the preset paging parameter, wherein the target page number is less than or equal to the total page number;
if the target page number is larger than the maximum allowable query page number, performing data query based on the preset screening condition, the maximum allowable query page number and an initial index, and determining the last data record in the queried data volume, wherein the initial index of the initial query is 1;
taking the next data record of the last data record as a new initial index, determining the difference between the target page number and the maximum allowable page number as a new target page number, and repeating the previous step and the current step until the target page number is less than or equal to the maximum allowable page number;
and performing data query based on the preset screening condition, the target page number and the initial index to obtain a query result.
2. The method of claim 1, wherein the using the next data record of the last data record as the new start index comprises:
and determining the next data record in the same direction as the sorting field in the last data record as a new initial index based on the preset sorting condition.
3. The method of claim 1, wherein the using the next data record of the last data record as the new start index comprises:
and determining the next data record in the same direction as the sorting field and the main key in the last data record as a new initial index based on the preset sorting condition and the preset main key sorting direction.
4. The method according to any one of claims 1 to 3, further comprising:
and returning the total data volume and the query result.
5. An apparatus for deep paging queries of data, the apparatus comprising:
the device comprises a first module, a second module and a third module, wherein the first module is used for determining total data volume meeting preset screening conditions based on an obtained data query instruction, and determining total page numbers corresponding to the total data volume based on preset paging parameters;
a second module, configured to obtain a target page number, and determine a maximum allowable query page number based on a preset deep paging minimum query data amount and the preset paging parameter, where the target page number is less than or equal to the total page number;
a third module, configured to query data based on the preset screening condition, the maximum allowed query page number, and an initial index if the target page number is greater than the maximum allowed query page number, and determine a last data record in the queried data volume, where the initial index of the initial query is 1;
a fourth module, configured to use a next data record of the last data record as a new start index, determine a difference between the target page number and the maximum allowable page number as a new target page number, and repeat operations of the third module and the present module until the target page number is less than or equal to the maximum allowable query page number;
and the fifth module is used for carrying out data query based on the preset screening condition, the target page number and the initial index to obtain a query result.
6. A computer-readable medium comprising, in combination,
stored thereon computer readable instructions to be executed by a processor to implement the method of any one of claims 1 to 4.
7. An apparatus for deep paging queries of data, the apparatus comprising:
one or more processors; and
memory storing computer readable instructions that, when executed, cause the processor to perform the operations of the method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110856921.8A CN113590623A (en) | 2021-07-28 | 2021-07-28 | Method, device and equipment for data deep paging query |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110856921.8A CN113590623A (en) | 2021-07-28 | 2021-07-28 | Method, device and equipment for data deep paging query |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113590623A true CN113590623A (en) | 2021-11-02 |
Family
ID=78251360
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110856921.8A Pending CN113590623A (en) | 2021-07-28 | 2021-07-28 | Method, device and equipment for data deep paging query |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113590623A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115098805A (en) * | 2022-08-25 | 2022-09-23 | 广州市千钧网络科技有限公司 | Method, device, equipment and storage medium for inquiring paging data information |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060149724A1 (en) * | 2005-01-03 | 2006-07-06 | Ritter Gerd M | Methods relating to data repository querying |
| CN1866251A (en) * | 2005-12-08 | 2006-11-22 | 华为技术有限公司 | Method and apparatus for reducing paging data retrieve time |
| CN105843956A (en) * | 2016-04-14 | 2016-08-10 | 北京搜狐新媒体信息技术有限公司 | Paging query method and system |
| CN106227894A (en) * | 2016-08-24 | 2016-12-14 | 中国农业银行股份有限公司 | A kind of data page querying method and device |
| CN108073661A (en) * | 2016-11-18 | 2018-05-25 | 北京京东尚科信息技术有限公司 | Data retrieval method and device, report generating system and method |
| CN109344197A (en) * | 2018-09-13 | 2019-02-15 | 广州帷策智能科技有限公司 | Paging method for down loading and device based on big data |
| WO2021031687A1 (en) * | 2019-08-16 | 2021-02-25 | 苏宁云计算有限公司 | Paging query method and apparatus for database sharding, and computer device |
-
2021
- 2021-07-28 CN CN202110856921.8A patent/CN113590623A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060149724A1 (en) * | 2005-01-03 | 2006-07-06 | Ritter Gerd M | Methods relating to data repository querying |
| CN1866251A (en) * | 2005-12-08 | 2006-11-22 | 华为技术有限公司 | Method and apparatus for reducing paging data retrieve time |
| CN105843956A (en) * | 2016-04-14 | 2016-08-10 | 北京搜狐新媒体信息技术有限公司 | Paging query method and system |
| CN106227894A (en) * | 2016-08-24 | 2016-12-14 | 中国农业银行股份有限公司 | A kind of data page querying method and device |
| CN108073661A (en) * | 2016-11-18 | 2018-05-25 | 北京京东尚科信息技术有限公司 | Data retrieval method and device, report generating system and method |
| CN109344197A (en) * | 2018-09-13 | 2019-02-15 | 广州帷策智能科技有限公司 | Paging method for down loading and device based on big data |
| WO2021031687A1 (en) * | 2019-08-16 | 2021-02-25 | 苏宁云计算有限公司 | Paging query method and apparatus for database sharding, and computer device |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115098805A (en) * | 2022-08-25 | 2022-09-23 | 广州市千钧网络科技有限公司 | Method, device, equipment and storage medium for inquiring paging data information |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11403346B2 (en) | Method and device for searching for information in applications | |
| US20170262361A1 (en) | Dynamic testing based on automated impact analysis | |
| EP3584704B1 (en) | Shared cache used to provide zero copy memory mapped database | |
| US20100287166A1 (en) | Method and system for search engine indexing and searching using the index | |
| US11288287B2 (en) | Methods and apparatus to partition a database | |
| US20180307736A1 (en) | Efficient Snapshot Generation of Data Tables | |
| CN108599973B (en) | Log association method, device and equipment | |
| US8055744B2 (en) | Resolution of group membership for resources | |
| CN111475105A (en) | Monitoring data storage method, device, server and storage medium | |
| CN108140022B (en) | Data query method and database system | |
| US8396858B2 (en) | Adding entries to an index based on use of the index | |
| CN111723245B (en) | Method for establishing association relation of different types of storage objects in data storage system | |
| CN113590623A (en) | Method, device and equipment for data deep paging query | |
| CN113868698A (en) | File desensitization method and equipment | |
| US10303687B2 (en) | Concurrent processing of data sources | |
| US11080299B2 (en) | Methods and apparatus to partition a database | |
| CN115391341A (en) | Distributed graph data processing system, method, device, equipment and storage medium | |
| CN105786478A (en) | Data processing method and device | |
| CN109543079B (en) | Data query method and device, computing equipment and storage medium | |
| CN112632211A (en) | Semantic information processing method and equipment for mobile robot | |
| CN109710833B (en) | Method and apparatus for determining content node | |
| CN112948389A (en) | MD 5-based database table data comparison method and equipment | |
| CN107122358B (en) | Hybrid query method and device | |
| CN113312382A (en) | Method, device and system for database paging query | |
| CN116701545A (en) | Audit data analysis method, audit data analysis device, storage medium and audit data processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |