CN113760972B - Data searching method and device - Google Patents
Data searching method and device Download PDFInfo
- Publication number
- CN113760972B CN113760972B CN202010530009.9A CN202010530009A CN113760972B CN 113760972 B CN113760972 B CN 113760972B CN 202010530009 A CN202010530009 A CN 202010530009A CN 113760972 B CN113760972 B CN 113760972B
- Authority
- CN
- China
- Prior art keywords
- data
- length
- step length
- service request
- duty ratio
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000012545 processing Methods 0.000 claims abstract description 19
- 238000012795 verification Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 9
- 238000013524 data verification Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000012796 concurrent verification Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 239000000835 fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 241000508269 Psidium Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data searching method and device, and relates to the technical field of computers. One embodiment of the method comprises the following steps: receiving a service request, and acquiring the effective data length requested by the service request; extracting a corresponding amount of data from a data source according to the current step length, and performing availability check on the data to obtain available data; if the number of the available data is greater than or equal to the effective data length, returning the data with the effective data length; otherwise, continuing to extract the data of the corresponding quantity of the next round from the data source according to the current step length, and carrying out availability check until the quantity of the available data is greater than or equal to the effective data length. According to the embodiment, the step length is dynamically adjusted by analyzing the data quality (availability), and the request processing can be carried out according to the step length after the dynamic adjustment, so that the searching efficiency of the data under different scenes and different quality is improved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data searching method and apparatus.
Background
Given the existing data sources, the effective data with the specified quantity is searched out according to the data arrangement sequence, currently, the industry usually takes out the first N pieces of data with fixed step length, and carries out availability check to screen out data which accords with expectations. If the screened data quantity does not meet the specified quantity, the next round of data (such as N+1-2N) needs to be continuously fetched for concurrency check.
In carrying out the present invention, the inventors have found that at least the following problems exist in the prior art:
The data source may vary in real time or with time periods and the data quality (availability) varies, and the fixed step size cannot adapt to the variation of the data, thereby reducing the efficiency of data searching. The single concurrent verification step length is fixed, and the effective data volume screened out is possibly not satisfied with the designated number due to the too short step length, so that the searching times are increased, and the data volume is excessively large, the data verification waste and the time consumption are increased due to the too long step length.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data searching method and apparatus, which at least can solve the problem that the step size in the prior art cannot be adaptively changed according to different data quality.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a data search method, including:
receiving a service request, and acquiring the effective data length requested by the service request;
Extracting a corresponding amount of data from a data source according to the current step length, and performing availability check on the data to obtain available data; the current step length is calculated in a preset mode;
If the number of the available data is greater than or equal to the effective data length, returning the data with the effective data length;
Otherwise, continuing to extract the data of the corresponding quantity of the next round from the data source according to the current step length, and carrying out availability check until the quantity of the available data is greater than or equal to the effective data length.
Optionally, the method further comprises: determining the execution times when the number of the available data is greater than or equal to the effective data length, and establishing a corresponding relation between the execution times and the service request; wherein the execution times are one or more times.
Optionally, the current step size is calculated in a predetermined manner, including:
Determining a processed service request set according to a preset period, acquiring the execution times of each service request after processing, and further counting the request quantity of which the execution times are one time;
Calculating the ratio of the request quantity in the total request quantity of the service request set;
If the duty ratio reaches a first duty ratio, the current step length is adjusted to be the difference between the last step length and a first numerical value;
If the duty ratio reaches a second duty ratio, the current step length is adjusted to be the sum of the last step length and a second numerical value; wherein the first duty cycle is greater than the second duty cycle.
Optionally, the method further comprises: determining unavailable data except available data in the extracted data, and marking the unavailable data based on a preset mark and the identification of the unavailable data; and
And pre-eliminating the unavailable data before the next round of data availability verification.
Optionally, after determining the unavailable data other than the available data in the extracted data, the method further includes: among the unavailable data, first unavailable data that is unavailable for all service requests is determined.
To achieve the above object, according to another aspect of the embodiments of the present invention, there is provided a data search device including:
The request receiving module is used for receiving a service request and acquiring the effective data length requested by the service request;
The data verification module is used for extracting corresponding data from a data source according to the current step length, and carrying out availability verification on the data to obtain available data; the current step length is calculated in a preset mode;
The result returning module is used for returning the data with the effective data length if the number of the available data is greater than or equal to the effective data length;
and the circulation processing module is used for continuously extracting the data of the corresponding quantity of the next round from the data source according to the current step length, and carrying out availability verification until the quantity of the available data is greater than or equal to the effective data length.
Optionally, the system further comprises a frequency counting module for: determining the execution times when the number of the available data is greater than or equal to the effective data length, and establishing a corresponding relation between the execution times and the service request; wherein the execution times are one or more times.
Optionally, the device further comprises a step adjustment module for:
Determining a processed service request set according to a preset period, acquiring the execution times of each service request after processing, and further counting the request quantity of which the execution times are one time;
Calculating the ratio of the request quantity in the total request quantity of the service request set;
If the duty ratio reaches a first duty ratio, the current step length is adjusted to be the difference between the last step length and a first numerical value;
If the duty ratio reaches a second duty ratio, the current step length is adjusted to be the sum of the last step length and a second numerical value; wherein the first duty cycle is greater than the second duty cycle.
Optionally, the method further comprises an unavailable marking module for: determining unavailable data except available data in the extracted data, and marking the unavailable data based on a preset mark and the identification of the unavailable data; and
And pre-eliminating the unavailable data before the next round of data availability verification.
Optionally, the unavailable marking module is further configured to: among the unavailable data, first unavailable data that is unavailable for all service requests is determined.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided a data search electronic device.
The electronic equipment of the embodiment of the invention comprises: one or more processors; and a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement any of the data lookup methods described above.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided a computer readable medium having stored thereon a computer program which, when executed by a processor, implements any of the above-described data search methods.
According to the solution provided by the present invention, one embodiment of the above invention has the following advantages or beneficial effects: the execution times of the processing requests are analyzed regularly to dynamically update the step length, and when the service requests are processed subsequently, the current step length can be extracted from the cache to search the data, so that the searching efficiency of the data under different scenes and different qualities is improved.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic flow chart of a data searching method according to an embodiment of the invention;
FIG. 2 is a flow chart of an alternative data lookup method according to an embodiment of the invention;
FIG. 3 is a flow diagram of a specific data lookup method according to an embodiment of the invention;
FIG. 4 is a schematic diagram of the main modules of a data searching apparatus according to an embodiment of the present invention;
FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
Fig. 6 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Referring to fig. 1, a main flowchart of a data searching method provided by an embodiment of the present invention is shown, including the following steps:
S101: receiving a service request, and acquiring the effective data length requested by the service request;
s102: extracting a corresponding amount of data from a data source according to the current step length, and performing availability check on the data to obtain available data; the current step length is calculated in a preset mode;
S103: if the number of the available data is greater than or equal to the effective data length, returning the data with the effective data length;
s104: otherwise, continuing to extract the data of the corresponding quantity of the next round from the data source according to the current step length, and carrying out availability check until the quantity of the available data is greater than or equal to the effective data length.
In the above embodiment, for steps S101 and S102, the server receives the service request sent by the program caller (e.g., consumer), and it is assumed that the effective data length of the request is N.
The current single seek step size is the optimal step size stored in the cache, and dynamically changes with the data quality (availability) of the data source and the number of data seeks, so that the step size used at this time may be different from the step size used at the next time, and the description is specifically referred to in the following fig. 2.
Assuming that the current step length is 2N, the TOP2N data in the existing data source is fetched, and the availability of the data is checked, whether the data is in stock or not, whether the user risk level accords with the data definition or not and the like. In most cases, TOPN filtered data cannot be used in its entirety, and multiple rounds of searching are generally required to be performed, so that the step size is not necessarily an integer multiple of N, but is necessarily greater than N.
It should be noted that, the data sources set in the method are ranked by scoring according to the data quality, and the service scene needs the optimal data, that is, the availability is verified one by one, so that the program efficiency can be improved by adopting concurrent verification, and the serial searching efficiency is relatively low.
For steps S103 and S104, for the data that is not available in the concurrent check result, a marking process, typically a specific mark+data ID, may be performed, and before the next data check, the marked data may be filtered out in advance, so as to reduce unnecessary concurrent check, and shorten the search time.
Further, the unavailable data may also be that all service requests of all service parties return an unavailable condition, such as that the total inventory of the data is insufficient and the data is put in a closed state, then the data does not need to be verified again for the service requests.
The marked unavailable data can be stored in a Guava cache, and the operation belongs to memory operation, so that network overhead is reduced, and program execution efficiency is improved.
If the number of available data obtained by verification does not meet N, the next round of data (such as 2N+1-4N) needs to be continuously taken out from the data source according to the current step length and concurrent verification is carried out until the number of available data obtained meets N.
Recording the searching times when the obtained number of the available data meets N, dividing the searching times into two types of successful one-time concurrent check and inquiry and successful multiple-time concurrent check and inquiry, and recording the round data executed when the searching is successful into a cache so as to adjust the step length according to the adaptability of the round data.
According to the method provided by the embodiment, the step length is dynamically adjusted by analyzing the data quality (availability), and the data can be extracted according to the step length after the dynamic adjustment, so that the searching efficiency of the data under different scenes and different quality is improved.
Referring to fig. 2, a flowchart of an alternative data searching method according to an embodiment of the present invention is shown, including the following steps:
s201: determining a processed service request set according to a preset period, acquiring the execution times of each service request after processing, and further counting the request quantity of which the execution times are one time;
s202: calculating the ratio of the request quantity in the total request quantity of the service request set;
S203: if the duty ratio reaches a first duty ratio, the current step length is adjusted to be the difference between the last step length and a first numerical value;
S204: if the duty ratio reaches a second duty ratio, the current step length is adjusted to be the sum of the last step length and a second numerical value; wherein the first duty cycle is greater than the second duty cycle.
In the above embodiment, for step S201, data is analyzed at a predetermined period (e.g., 10 minutes, configurable) at regular time, and a concurrent check is determined to obtain the step size of the specified data N as the optimal step size.
For each request, after the searching result meets the specified length, recording the round data executed when the searching is successful into a cache; the data are classified into two types, namely one concurrent check query success and multiple concurrent check queries success.
Therefore, the service request set which is processed in the period and the execution times of each request can be acquired according to the preset period, and then the request of 'one concurrent check query success' is extracted from the service request set.
For steps S202-S204, the number of extracted requests is counted, and the ratio of the number to the total number of requests in the present period (here, 2N is taken as an example) is calculated:
1) If the duty cycle reaches the first duty cycle (the peak value of the one-time searching success rate is considered), 2N-1 operation is carried out (1 is only an example here), and the single concurrency checking step length is reduced;
2) If the duty cycle drops to a second duty cycle (assuming a dip in one lookup success rate), 2n+1 operations (1 is only an example here), a single concurrent check step is added to increase one lookup success rate.
The optimal step length stored in the cache can be obtained in real time in the subsequent data searching process, so that the data searching efficiency is improved. In addition, after the step size is updated, the data in the cache may be emptied, for example, the count information of the successful execution of the data search is not the marking data. For example, consumer requests 100 times within 10 minutes, wherein 35 times of execution acquire data with a specified length, the rest 65 times all require 2 or more times, and the data is emptied after data analysis.
According to the method provided by the embodiment, the step length is updated dynamically at regular intervals, and when the service request is processed later, the dynamic value can be obtained from the cache in real time to perform data length processing, so that data searching is realized flexibly, and the searching efficiency is improved.
Referring to fig. 3, a flowchart of a specific data searching method according to an embodiment of the present invention is shown, including the following steps:
S301: receiving a service request, and acquiring the effective data length requested by the service request;
s302: extracting a corresponding amount of data from a data source according to the current step length, and performing availability check on the data to obtain available data;
s303: determining unavailable data except available data in the extracted data, and marking the unavailable data based on a preset mark and the identification of the unavailable data;
s304: if the number of the available data is greater than or equal to the effective data length, returning the data with the effective data length;
S305: otherwise, continuing to extract the data of the corresponding quantity of the next round from the data source according to the current step length, and performing availability verification until the quantity of the available data is greater than or equal to the effective data length;
s306: determining the execution times when the number of the available data is greater than or equal to the effective data length, and establishing a corresponding relation between the execution times and the service request; wherein the execution times are one or more times;
S307: determining a processed service request set according to a preset period, acquiring the execution times of each service request after processing, and further counting the request quantity of which the execution times are one time;
s308: calculating the ratio of the request quantity in the total request quantity of the service request set;
S309: if the duty ratio reaches the first duty ratio, the current step length is adjusted to be the difference between the last step length and the first numerical value;
S310: if the duty ratio reaches the second duty ratio, the current step length is adjusted to be the sum of the last step length and the second numerical value; wherein the first duty cycle is greater than the second duty cycle.
The method provided by the embodiment of the invention periodically analyzes the execution times of the processing request to dynamically update the step length, and can extract the current step length from the cache to search the data when the service request is processed subsequently, thereby improving the searching efficiency of the data under different scenes and different qualities.
Referring to fig. 4, a schematic diagram of main modules of a data searching apparatus 400 according to an embodiment of the present invention is shown, including:
A request receiving module 401, configured to receive a service request, and obtain an effective data length requested by the service request;
the data verification module 402 is configured to extract a corresponding amount of data from a data source according to a current step size, and perform availability verification on the data to obtain available data; the current step length is calculated in a preset mode;
a result returning module 403, configured to return data of the valid data length if the number of available data is greater than or equal to the valid data length;
And the circulation processing module 404 is configured to continue to extract a corresponding amount of data of the next round from the data source according to the current step size, and perform availability verification until the amount of available data is greater than or equal to the valid data length.
The implementation device of the present invention further includes a frequency statistics module 405 (not shown in the figure) for: determining the execution times when the number of the available data is greater than or equal to the effective data length, and establishing a corresponding relation between the execution times and the service request; wherein the execution times are one or more times.
The implementation device of the present invention further includes a step adjustment module 406 (not shown in the figure) for:
Determining a processed service request set according to a preset period, acquiring the execution times of each service request after processing, and further counting the request quantity of which the execution times are one time;
Calculating the ratio of the request quantity in the total request quantity of the service request set;
If the duty ratio reaches a first duty ratio, the current step length is adjusted to be the difference between the last step length and a first numerical value;
If the duty ratio reaches a second duty ratio, the current step length is adjusted to be the sum of the last step length and a second numerical value; wherein the first duty cycle is greater than the second duty cycle.
The implementation device of the present invention further comprises an unavailable marking module 407 (not shown in the figure) for: determining unavailable data except available data in the extracted data, and marking the unavailable data based on a preset mark and the identification of the unavailable data; and
And pre-eliminating the unavailable data before the next round of data availability verification.
In the implementation apparatus of the present invention, the unavailable marking module 407 is further configured to: among the unavailable data, first unavailable data that is unavailable for all service requests is determined.
In addition, the implementation of the apparatus in the embodiments of the present invention has been described in detail in the above method, so that the description is not repeated here.
Fig. 5 illustrates an exemplary system architecture 500 in which embodiments of the present invention may be applied.
As shown in fig. 5, the system architecture 500 may include terminal devices 501, 502, 503, a network 504, and a server 505 (by way of example only). The network 504 is used as a medium to provide communication links between the terminal devices 501, 502, 503 and the server 505. The network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 505 via the network 504 using the terminal devices 501, 502, 503 to receive or send messages or the like. Various communication client applications can be installed on the terminal devices 501, 502, 503.
The terminal devices 501, 502, 503 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 505 may be a server providing various services, such as a background management server (by way of example only) providing support for shopping-type websites browsed by users using the terminal devices 501, 502, 503.
It should be noted that, the method provided by the embodiment of the present invention is generally performed by the server 505, and accordingly, the apparatus is generally disposed in the server 505.
It should be understood that the number of terminal devices, networks and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 6, there is illustrated a schematic diagram of a computer system 600 suitable for use in implementing an embodiment of the present invention. The terminal device shown in fig. 6 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 601.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor includes a request receiving module, a data checking module, a result returning module, and a loop processing module. The names of these modules do not constitute a limitation on the module itself in some cases, and the result return module may also be described as a "return result module", for example.
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include:
receiving a service request, and acquiring the effective data length requested by the service request;
Extracting a corresponding amount of data from a data source according to the current step length, and performing availability check on the data to obtain available data; the current step length is calculated in a preset mode;
If the number of the available data is greater than or equal to the effective data length, returning the data with the effective data length;
Otherwise, continuing to extract the data of the corresponding quantity of the next round from the data source according to the current step length, and carrying out availability check until the quantity of the available data is greater than or equal to the effective data length.
According to the technical scheme of the embodiment of the invention, the execution times of the processing request are analyzed regularly to dynamically update the step length, and when the service request is processed subsequently, the current step length can be extracted from the cache to search the data, so that the searching efficiency of the data under different scenes and different qualities is improved.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (8)
1. A method for searching data, comprising:
receiving a service request, and acquiring the effective data length requested by the service request;
Extracting a corresponding amount of data from a data source according to the current step length, and performing availability check on the data to obtain available data; the current step length is calculated in a preset mode, and the method comprises the following steps: determining a processed service request set according to a preset period, acquiring the execution times of each service request after processing, and further counting the request quantity of which the execution times are one time; calculating the ratio of the request quantity in the total request quantity of the service request set; if the duty ratio reaches a first duty ratio, the current step length is adjusted to be the difference between the last step length and a first numerical value; if the duty ratio reaches a second duty ratio, the current step length is adjusted to be the sum of the last step length and a second value, and the first duty ratio is larger than the second duty ratio;
If the number of the available data is greater than or equal to the effective data length, returning the data with the effective data length;
Otherwise, continuing to extract the data of the corresponding quantity of the next round from the data source according to the current step length, and carrying out availability check until the quantity of the available data is greater than or equal to the effective data length.
2. The method as recited in claim 1, further comprising:
determining the execution times when the number of the available data is greater than or equal to the effective data length, and establishing a corresponding relation between the execution times and the service request; wherein the execution times are one or more times.
3. The method as recited in claim 1, further comprising:
Determining unavailable data except available data in the extracted data, and marking the unavailable data based on a preset mark and the identification of the unavailable data; and
And pre-eliminating the unavailable data before the next round of data availability verification.
4. A method according to claim 3, further comprising, after said determining unavailable data other than available data in the extracted data:
among the unavailable data, first unavailable data that is unavailable for all service requests is determined.
5. A data search device, comprising:
The request receiving module is used for receiving a service request and acquiring the effective data length requested by the service request;
The data verification module is used for extracting corresponding data from a data source according to the current step length, and carrying out availability verification on the data to obtain available data; the current step length is calculated in a preset mode, and the method comprises the following steps: determining a processed service request set according to a preset period, acquiring the execution times of each service request after processing, and further counting the request quantity of which the execution times are one time; calculating the ratio of the request quantity in the total request quantity of the service request set; if the duty ratio reaches a first duty ratio, the current step length is adjusted to be the difference between the last step length and a first numerical value; if the duty ratio reaches a second duty ratio, the current step length is adjusted to be the sum of the last step length and a second value, and the first duty ratio is larger than the second duty ratio;
The result returning module is used for returning the data with the effective data length if the number of the available data is greater than or equal to the effective data length;
and the circulation processing module is used for continuously extracting the data of the corresponding quantity of the next round from the data source according to the current step length, and carrying out availability verification until the quantity of the available data is greater than or equal to the effective data length.
6. The apparatus of claim 5, further comprising a count module configured to:
determining the execution times when the number of the available data is greater than or equal to the effective data length, and establishing a corresponding relation between the execution times and the service request; wherein the execution times are one or more times.
7. An electronic device, comprising:
one or more processors;
Storage means for storing one or more programs,
When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-4.
8. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010530009.9A CN113760972B (en) | 2020-06-11 | 2020-06-11 | Data searching method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010530009.9A CN113760972B (en) | 2020-06-11 | 2020-06-11 | Data searching method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113760972A CN113760972A (en) | 2021-12-07 |
| CN113760972B true CN113760972B (en) | 2024-05-17 |
Family
ID=78785357
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010530009.9A Active CN113760972B (en) | 2020-06-11 | 2020-06-11 | Data searching method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113760972B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019205382A1 (en) * | 2018-04-28 | 2019-10-31 | 平安科技（深圳）有限公司 | Electronic device, credit investigation data acquisition method, and storage medium |
| CN110851419A (en) * | 2018-08-03 | 2020-02-28 | 北京京东金融科技控股有限公司 | Data migration method and device |
-
2020
- 2020-06-11 CN CN202010530009.9A patent/CN113760972B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019205382A1 (en) * | 2018-04-28 | 2019-10-31 | 平安科技（深圳）有限公司 | Electronic device, credit investigation data acquisition method, and storage medium |
| CN110851419A (en) * | 2018-08-03 | 2020-02-28 | 北京京东金融科技控股有限公司 | Data migration method and device |
Non-Patent Citations (2)
| Title |
|---|
| Ghannadpour, SF等.Multiobjective Dynamic Vehicle Routing Problem With Fuzzy Travel Times and Customers' Satisfaction in Supply Chain Management.《IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT》.2013,全文. * |
| 基于TCAM的K步长多模式匹配算法及硬件实现;赵海斌;李训根;王奇敏;;微电子学与计算机;20130405(04);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113760972A (en) | 2021-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11146502B2 (en) | Method and apparatus for allocating resource | |
| CN109388626B (en) | Method and apparatus for assigning numbers to services | |
| CN108984553B (en) | Caching method and device | |
| CN110321252B (en) | Skill service resource scheduling method and device | |
| CN110909022A (en) | Data query method and device | |
| CN110866040A (en) | User portrait generation method, device and system | |
| CN111435380B (en) | Page cross-domain interaction method, system, device and storage device | |
| CN113312553B (en) | User tag determining method and device | |
| CN113760982B (en) | Data processing method and device | |
| CN111062572A (en) | Task allocation method and device | |
| CN112685481B (en) | Data processing method and device | |
| CN113760972B (en) | Data searching method and device | |
| CN112948138A (en) | Method and device for processing message | |
| CN116204428A (en) | Test case generation method and device | |
| CN112784139B (en) | Query method, device, electronic equipment and computer readable medium | |
| CN112783914B (en) | Method and device for optimizing sentences | |
| CN113138943B (en) | Method and device for processing request | |
| CN112783615B (en) | Data processing task cleaning method and device | |
| CN113434754A (en) | Method and device for determining recommended API (application program interface) service, electronic equipment and storage medium | |
| CN113448652A (en) | Request processing method and device | |
| CN112862554A (en) | Order data processing method and device | |
| CN113722193A (en) | Method and device for detecting page abnormity | |
| CN109213815B (en) | Method, device, server terminal and readable medium for controlling execution times | |
| CN113779048A (en) | Data processing method and device | |
| CN113778657B (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |