CN113760968A - Data query method, device, system, electronic equipment and storage medium - Google Patents
Data query method, device, system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113760968A CN113760968A CN202011018899.1A CN202011018899A CN113760968A CN 113760968 A CN113760968 A CN 113760968A CN 202011018899 A CN202011018899 A CN 202011018899A CN 113760968 A CN113760968 A CN 113760968A
- Authority
- CN
- China
- Prior art keywords
- fragmentation
- data query
- information
- fragmentation information
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 97
- 238000013467 fragmentation Methods 0.000 claims abstract description 283
- 238000006062 fragmentation reaction Methods 0.000 claims abstract description 283
- 239000012634 fragment Substances 0.000 claims description 27
- 230000015654 memory Effects 0.000 claims description 23
- 238000012545 processing Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 abstract description 11
- 238000004891 communication Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data query method, a device, a system, an electronic device and a storage medium, comprising: receiving a data query request input by a user, receiving first fragmentation information aiming at the data query request sent by a server, generating second fragmentation information corresponding to the clients according to the data query request and the total number of the clients performing the data query request, acquiring and outputting a data query result from a preset database according to the first fragmentation information and the second fragmentation information, in this embodiment, the client may perform fragmentation (which may be understood as secondary fragmentation) for the data query request, and may perform corresponding data query operation by combining the first fragmentation information sent by the server and the second fragmentation information generated by its own fragmentation, fully considering the number of clients actually performing the data query task, the reliability and the accuracy of data query can be improved, and the technical effect of network resource consumption between the client and the server is reduced.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data query method, apparatus, system, electronic device, and storage medium.
Background
With the development of internet technology and the continuous increase of information quantity, how to realize safe and efficient data query becomes a problem to be solved urgently.
In the prior art, data query is generally implemented by using a server and a client mode, for example, the server schedules tasks, and the client performs specific task execution according to the scheduling of the server. For example, the server configures tasks, such as a task name, an execution period, a total fragmentation number, and the like, and fragments the total fragmentation number according to a certain fragmentation policy, and the client performs a data query operation corresponding to the fragmentation information based on the fragmentation information fragmented by the server.
However, in the process of implementing the present application, the inventors found that at least the following technical problems exist: the fragmentation information is determined by the server based on the total fragmentation number of the fixed configuration, and the number of the clients may be changed, thereby possibly causing a problem that the reliability of the result of the data query is low.
Disclosure of Invention
The embodiment of the application provides a data query method, a data query device, a data query system, electronic equipment and a storage medium, and is used for solving the technical problem that the reliability of a data query result is low.
In a first aspect, an embodiment of the present application provides a data query method, where the method includes:
receiving a data query request input by a user, and receiving first fragmentation information aiming at the data query request sent by a server;
generating second fragmentation information corresponding to the clients according to the data query request and the total number of the clients executing the data query request;
and acquiring and outputting a data query result from a preset database according to the first fragmentation information and the second fragmentation information.
In this embodiment, the client may perform fragmentation (which may be understood as secondary fragmentation) for the data query request, and may perform corresponding data query operations by combining the first fragmentation information sent by the server and the second fragmentation information generated by its own fragmentation, fully considering the number of clients actually performing the data query task, which may improve reliability and accuracy of data query, and reduce technical effects of network resource consumption between the client and the server.
In some embodiments, the data query request carries data information to be queried, and the generating second fragmentation information according to the data query request and the total number of clients executing the data query request includes:
generating a data query task according to the data information to be queried;
and carrying out fragmentation processing on the data query data task according to the total number to generate second fragmentation information.
In some embodiments, the data query task includes a field value, and the generating a data query task according to the data information to be queried includes:
determining a field name corresponding to the data query request according to the data information to be queried;
generating the field value corresponding to the field name.
In some embodiments, the fragmenting the data query data task according to the total number, and generating second fragmentation information includes:
carrying out fragmentation processing on the data query task according to a preset fragmentation strategy, the field value and the total number to generate second fragmentation information;
wherein the fragmentation strategy comprises: at least one of a random fragmentation method, an average fragmentation method, and a remainder fragmentation method.
In some embodiments, the fragmentation policy is the same as a fragmentation policy generated by the server to generate the first fragmentation information according to the data query request.
In this embodiment, the client generates the second fragmentation information by using the same distribution policy as that used by the server to generate the first fragmentation information, so that the first fragmentation information and the second fragmentation information can be in the same fragmentation method, and thus the first fragmentation information and the second fragmentation information have higher comparability, thereby achieving the technical effects of reliability and accuracy of data query.
In some embodiments, if the fragmentation policy is the remainder fragmentation method, performing fragmentation processing on the data query task according to the remainder fragmentation method, the field value, and the total number, and generating the second fragmentation information includes:
and performing remainder processing on the field value according to the total number to generate the second fragmentation information.
In some embodiments, the obtaining and outputting a data query result from a preset database according to the first fragmentation information and the second fragmentation information includes:
determining common fragmentation information of the first fragmentation information and the second fragmentation information;
and acquiring and outputting the data query result from the database according to the common fragmentation information.
In some embodiments, the first fragmentation information comprises a set of first fragmentation entries, the second fragmentation information comprises a set of second fragmentation entries, and the determining fragmentation information common to the first fragmentation information and the second fragmentation information comprises:
determining an intersection of the set of first sharded items and the set of second sharded items;
and the obtaining and outputting the data query result from the database according to the common fragmentation information comprises: and acquiring and outputting the data query result from the database according to the fragment items in the intersection.
In this embodiment, the client performs data query in the database according to the fragment items in the intersection, and generates a data query result, where the fragment items in the intersection are determined based on the number of total clients executing the data query task, and therefore, it is equivalent to determining a part of tasks in the data query task that the client needs to execute based on the current resources of the client, thereby achieving the technical effects of completeness and comprehensiveness of processing the data query task.
In a second aspect, an embodiment of the present application provides a client, where the client includes:
the receiving module is used for receiving a data query request input by a user and receiving first fragmentation information aiming at the data query request sent by a server;
the generating module is used for generating second fragmentation information corresponding to the clients according to the data query requests and the total number of the clients executing the data query requests;
the acquisition module is used for acquiring a data query result from a preset database according to the first fragmentation information and the second fragmentation information;
and the output module is used for outputting the data query result.
In some embodiments, the data query request carries data information to be queried, and the generating module is configured to generate a data query task according to the data information to be queried, perform fragmentation processing on the data query task according to the total amount, and generate second fragmentation information.
In some embodiments, the generating module is configured to perform fragmentation processing on the data query task according to a preset fragmentation policy, the field value, and the total number, and generate the second fragmentation information;
wherein the fragmentation strategy comprises: at least one of a random fragmentation method, an average fragmentation method, and a remainder fragmentation method.
In some embodiments, the fragmentation policy is the same as a fragmentation policy generated by the server to generate the first fragmentation information according to the data query request.
In some embodiments, if the fragmentation policy is the remainder fragmentation method, the generation module is configured to perform remainder processing on the field value according to the total number to generate the second fragmentation information.
In some embodiments, the obtaining module is configured to determine common fragmentation information of the first fragmentation information and the second fragmentation information, and obtain the data query result from the database according to the common fragmentation information.
In some embodiments, the first sharding information includes a set of first sharding items, the second sharding information includes a set of second sharding items, and the obtaining module is configured to determine an intersection of the set of first sharding items and the set of second sharding items, and obtain the data query result from the database according to the sharding items in the intersection.
In a third aspect, an embodiment of the present application provides a data query system, where the system includes:
the server is used for generating and sending first fragmentation information to the client according to the received data query request:
a client as in any preceding embodiment.
In a fourth aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;
a memory for storing the processor-executable instructions;
wherein the processor, when executing the instructions in the memory, is configured to implement the method of any of the embodiments described above.
In a fifth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the method according to any of the above embodiments.
The embodiment of the application provides a data query method, a device, a system, an electronic device and a storage medium, comprising: receiving a data query request input by a user, receiving first fragmentation information aiming at the data query request sent by a server, generating second fragmentation information corresponding to a client according to the data query request and the total number of the clients executing the data query request, acquiring and outputting a data query result from a preset database according to the first fragmentation information and the second fragmentation information, in the related art, fragmenting a data query task by the server, sending corresponding fragmentation information (i.e. the first fragmentation information in the embodiment) to the client, executing corresponding data query operation by the client based on the fragmentation information (i.e. the first fragmentation information in the embodiment), while in the embodiment, the client can perform fragmentation (can be understood as secondary fragmentation) aiming at the data query request and can combine the first fragmentation information sent by the server and the second fragmentation information generated by the self-fragmentation, the corresponding data query operation is executed, the number of the clients actually executing the data query task is fully considered, the reliability and the accuracy of the data query can be improved, and the technical effect of network resource consumption between the clients and the server is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a schematic diagram of an application scenario of a data query method according to an embodiment of the present application;
FIG. 2 is a flow chart illustrating a data query method according to an embodiment of the present application;
FIG. 3 is a schematic flow chart illustrating a data query method according to another embodiment of the present application;
FIG. 4 is a schematic flow chart illustrating a data query method according to another embodiment of the present application;
FIG. 5 is a schematic diagram of a client according to an embodiment of the present application;
fig. 6 is a block diagram of an electronic device according to an embodiment of the present application.
With the foregoing drawings in mind, certain embodiments of the disclosure have been shown and described in more detail below. These drawings and written description are not intended to limit the scope of the disclosed concepts in any way, but rather to illustrate the concepts of the disclosure to those skilled in the art by reference to specific embodiments.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Referring to fig. 1, fig. 1 is a schematic view of an application scenario of a data query method according to an embodiment of the present application.
As shown in fig. 1, the application scenario includes a server 100, a Zookeeper 200, and a client 300.
The server 100 may configure the data query task, such as a data query task name, an execution period, a total number of fragments, and the like. The server 100 may preset a fragmentation policy, fragment the data query task based on the fragmentation policy and the total fragmentation number, generate a fragmentation item, and send the fragmentation item to the Zookeeper 200.
Specifically, the sharding item may be understood as sharding content obtained by sharding the data query task by the server, that is, a certain part of the data query task that needs to be executed. That is to say, the server can split one data query task into a plurality of tasklets, so that each client executes the corresponding tasklet, thereby completing the whole data query task.
For example, the data query task is to traverse a certain table in the database, and as shown in fig. 1, there are 3 clients 300 in total, and the total number of shards is 9, the server shards the task traversing the table, and waits until the shards are 0, 1, 2, 3, 4, 5, 6, 7, and 8 respectively, that is, splits traversing the table into 9 small tasks, sends the 9 shards to the Zookeeper 200, and is distributed to the 3 clients 300 by the Zookeeper 200.
The Zookeeper 200 is a reliable coordination system of a distributed system, and may specifically be a distributed, open-source distributed application coordination service, and the Zookeeper 200 may be integrated with the server 100.
In this embodiment, the Zookeeper 200 may receive and store the sharding item transmitted by the server 100, and may transmit the sharding item to each client 300.
The client 300 is configured to obtain the sharding items from the Zookeeper 200, and traverse the table contents corresponding to the respective sharding items from the database according to the sharding items.
It should be noted that the above application scenarios are only used for exemplarily illustrating application scenarios to which the data query method according to the embodiment of the present application may be applied, and are not to be construed as a limitation to the application scenarios. For example, in some possible implementation schemes, elements may be adaptively added on the basis of the application scenario as shown in fig. 1, such as increasing the number of clients or adding new elements; elements can also be adaptively reduced on the basis as shown in fig. 1, such as reducing the number of clients, and so on.
It can be understood that the server may configure the data query task, such as the total number of fragments of the data query task, etc. in the above embodiment, however, in the actual application process, the number of the clients may be dynamically changed, for example, due to the service requirement, a certain number of clients are added; for another example, in order to save resources, a certain number of clients are reduced; as another example, at least one client goes down, and so on, and when the number of clients changes, the sharding items allocated to the clients by the server cannot completely include the content of the data query task, thereby possibly causing a problem that the reliability of the result of the data query is low.
The inventor of the present application has obtained the inventive concept of the present application after creative efforts: the client performs secondary fragmentation on the data query task on the basis of receiving fragmentation information (such as the fragmentation items in the embodiments) of the primary fragmentation of the server, and determines a data query result based on the received fragmentation information and the fragmentation information obtained by the fragmentation of the client.
The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
According to an aspect of the embodiments of the present application, the embodiments of the present application provide a data query method, which is applied to a client, and the method may be applied to an application scenario as shown in fig. 1.
Referring to fig. 2, fig. 2 is a schematic flow chart illustrating a data query method according to an embodiment of the present application.
As shown in fig. 2, the method includes:
s101: the method comprises the steps of receiving a data query request input by a user, and receiving first fragment information aiming at the data query request sent by a server.
The execution subject of the embodiment of the present application may be a data query device, and the data query device may specifically be a client, and may also be a processor or a chip, etc.
For example, when the data query method of the present embodiment is applied to the application scenario shown in fig. 1, the data query device may be a client as shown in fig. 1.
The first fragmentation information may be used to characterize related information obtained by fragmentation by the server according to the data query request, for example, the first fragmentation information may include a fragmentation item as described in the above embodiments.
S102: and generating second fragmentation information corresponding to the clients according to the data query requests and the total number of the clients executing the data query requests.
Similarly, the second fragmentation information may be used to characterize related information obtained by fragmenting the data query request by the client, and the related information is for the client, for example, the second fragmentation information may include a fragmentation item as described in the above embodiments.
That is to say, in this embodiment, on one hand, the client may receive the first fragmentation information sent by the server, and on the other hand, the client may further fragment the data query request, and specifically, obtain the second fragmentation information based on the total number of clients performing the data query request.
It is worth noting that, in the embodiment, when the client generates the second fragmentation information corresponding to the client, the total number of the clients executing the data query request is considered, so that the problem that the integrity of the completed data query request is low due to the change of the total number of the clients executing the data query request in the related art is avoided, and the technical effects of reliability, accuracy and comprehensiveness of the data query are improved.
S103: and acquiring and outputting a data query result from a preset database according to the first fragment information and the second fragment information.
It is worth noting that, in this embodiment, the client acquires and outputs the data query result based on the first fragmentation information and the second fragmentation information, compared to the related art that the data query result is acquired and output based on only the first fragmentation information, because the second fragmentation information in this embodiment is determined by the total number of clients that perform the data query request, the problem that the reliability of the result of the data query is low due to the fact that the number of clients participating in performing the data query request may be a variable number can be avoided, thereby achieving the technical effect of improving the reliability of the result of the data query, and generating the second fragmentation information by the client can reduce the network interaction between the client and the server, thereby achieving the technical effects of saving network resources and improving the data query efficiency.
Based on the above analysis, an embodiment of the present application provides a data query method, which can be applied to a client, and the method includes: receiving a data query request input by a user, receiving first fragmentation information aiming at the data query request sent by a server, generating second fragmentation information corresponding to a client according to the data query request and the total number of the clients executing the data query request, acquiring and outputting a data query result from a preset database according to the first fragmentation information and the second fragmentation information, in the related art, fragmenting a data query task by the server, sending corresponding fragmentation information (i.e. the first fragmentation information in the embodiment) to the client, executing corresponding data query operation by the client based on the fragmentation information (i.e. the first fragmentation information in the embodiment), while in the embodiment, the client can perform fragmentation (can be understood as secondary fragmentation) aiming at the data query request and can combine the first fragmentation information sent by the server and the second fragmentation information generated by the self-fragmentation, the corresponding data query operation is executed, the number of the clients actually executing the data query task is fully considered, the reliability and the accuracy of the data query can be improved, and the technical effect of network resource consumption between the clients and the server is reduced.
For the reader to understand the data query method provided by the present application more thoroughly, the data query method of the present application is now described in more detail with reference to the dimension of generating the second piece of information from the client in fig. 3. Fig. 3 is a schematic flow chart of a data query method according to another embodiment of the present application.
As shown in fig. 3, the method includes:
s201: the method comprises the steps of receiving a data query request input by a user, and receiving first fragment information aiming at the data query request sent by a server, wherein the data query request carries data information to be queried.
For a part of the description about S201, reference may be made to S101, which is not described herein again.
In this embodiment, the data query request carries data information that the user desires to query (i.e., data information to be queried). The data information to be queried may be a table, and if the user wishes to perform a traversal on a table in the database as described in the above embodiment, the data information to be queried may be the table; the data information to be queried may also be a document, etc., which are not listed here.
S202: and generating a data query task according to the data information to be queried.
In one possible implementation, S202 may include:
s2021: and determining the field name corresponding to the data query request according to the data information to be queried.
When the client side obtains the data information to be queried, a field corresponding to the data information to be queried can be determined, and a field name corresponding to the field is determined, wherein the field name is a field name corresponding to the data query request.
S2022: and generating a field value corresponding to the field name, wherein the data query task comprises the field value.
The client may generate a field value corresponding to the field name based on a java reflection technology, or may generate the field value of the field name based on an Object Navigation Language (OGNL) technology.
S203: and carrying out fragmentation processing on the data query data task according to the total quantity to generate second fragmentation information.
In one possible implementation, S203 may include: and carrying out fragmentation processing on the data query task according to a preset fragmentation strategy, a preset field value and the total amount to generate second fragmentation information.
Wherein, the slicing strategy comprises: at least one of a random fragmentation method, an average fragmentation method, and a remainder fragmentation method.
Now, taking the fragmentation strategy as a random fragmentation method as an example, the method for generating the second fragmentation information by the client is explained as follows:
and randomly fragmenting the field value according to the total number to obtain fragmentation information corresponding to each client, and determining the fragmentation information corresponding to the client as second fragmentation information.
For example, with reference to the embodiment shown in fig. 1, the total number is 3, the total number of fragments may be determined according to a field value, for example, the field value may be determined as the total number of fragments, or the total number of fragments may be determined as a multiple of the total number according to the field value, if the total number of fragments is 9, after the client performs fragmentation by using a random fragmentation method, fragmentation information of a certain client is [ 0, 1, 4 ], fragmentation information of another client is [ 2, 3, 5 ], fragmentation information of another client is [ 6, 7, 8 ], and if the fragmentation information corresponding to the client is [ 2, 3, 5 ], then [ 2, 3, 5 ] is the second fragmentation information.
Based on the above analysis, the fragmentation policy may further include an average fragmentation method, and now taking the fragmentation policy as the average fragmentation method as an example, the method for generating the second fragmentation information by the client is explained as follows:
and averagely fragmenting the field values according to the total number to obtain fragmentation information corresponding to each client, and determining the fragmentation information corresponding to the client as second fragmentation information.
For example, with reference to the embodiment shown in fig. 1, the total number is 3, the total number of slices may be determined according to a field value, for example, the field value may be determined as the total number of slices, or the total number of slices may be determined as a multiple of the total number according to the field value, if the total number of slices is 9, after a client performs slicing by using an average slicing method, the slicing information of a certain client is [ 0, 1, 2 ], the slicing information of another client is [ 3, 4, 5 ], the slicing information of another client is [ 6, 7, 8 ], and if the slicing information corresponding to the client is [ 3, 4, 5 ], the [ 3, 4, 5 ] is the second slicing information.
Based on the above analysis, the fragmentation strategy may further include a remainder fragmentation method, and now taking the fragmentation strategy as the remainder fragmentation method as an example, the method for generating the second fragmentation information by the client is explained as follows:
and taking the rest of the field values according to the total number, fragmenting to obtain fragmentation information corresponding to each client, and determining the fragmentation information corresponding to the client as second fragmentation information.
For example, with reference to the embodiment shown in fig. 1, the total number is 3, the total number of slices may be determined according to a field value, for example, the field value may be determined as the total number of slices, or the total number of slices may be determined as a multiple of the total number according to the field value, if the total number of slices is 9, after the client performs slicing by using a remainder-taking slicing method (for example, remainder is taken by 3), the slicing information of one client is [ 0, 3, 6 ], the slicing information of another client is [ 1, 4, 7 ], the slicing information of another client is [ 2, 5, 8 ], and if the slicing information corresponding to the client is [ 1, 4, 7 ], then [ 1, 4, 7 ] is the second slicing information.
In a possible implementation scheme, the fragmentation policy adopted by the client is the same as the fragmentation policy for the server to generate the first fragmentation information.
Based on the above embodiment, it can be seen that the first fragmentation information is generated by the server according to the fragmentation policy of the device in advance, the second fragmentation information is generated by the client using at least one of the three fragmentation policies, and the client generates the second fragmentation information by using the same distribution policy as that used by the server to generate the first fragmentation information, so that the first fragmentation information and the second fragmentation information can be in the same fragmentation method, and thus the first fragmentation information and the second fragmentation information have higher comparability, thereby achieving the technical effects of reliability and accuracy of data query.
S204: and acquiring and outputting a data query result from a preset database according to the first fragment information and the second fragment information.
For the description of S204, reference may be made to S103, which is not described herein again.
For the reader to understand the data query method provided by the present application more thoroughly, the data query method of the present application is described in more detail with reference to fig. 4, where the dimension of the data query result generated from the client according to the first fragment information and the second fragment information is used. Fig. 4 is a schematic flow chart of a data query method according to another embodiment of the present application.
As shown in fig. 4, the method includes:
s301: the method comprises the steps of receiving a data query request input by a user, and receiving first fragment information aiming at the data query request sent by a server.
For the description of S301, refer to S101 or S201, which is not described herein again.
S302: and generating second fragmentation information corresponding to the clients according to the data query requests and the total number of the clients executing the data query requests.
For the description of S302, refer to S102, or refer to S202 to S203, which is not described herein again.
S303: common fragmentation information of the first fragmentation information and the second fragmentation information is determined.
In this step, the client determines the fragmentation information included in both the first fragmentation information and the second fragmentation information, and may refer to the partial fragmentation information as common fragmentation information.
In combination with the above analysis, in a possible implementation, the first fragmentation information may include a plurality of fragmentation items, and the second fragmentation information may include a plurality of fragmentation items, that is, the first fragmentation information includes a set of first fragmentation items, and the second fragmentation information includes a set of second fragmentation items, and then the common fragmentation information may be an intersection of the set of first fragmentation items and the set of second fragmentation items.
S304: and acquiring and outputting a data query result from the database according to the common fragment information.
In this step, on the premise that the client determines the intersection, the client may obtain and output the data query result from the database according to the fragmentation items in the intersection.
It should be noted that, since the data query result is determined by the client based on the fragment items in the intersection, which is equivalent to that the client comprehensively considers the fragment item provided by the server and the fragment item generated by itself, and the fragment item generated by itself is determined based on the client currently executing the data query task, which is equivalent to that the total resource condition of the client executing the data query task is considered, the technical effects of fully utilizing the resources of each client and improving the reliability and accuracy of the data query can be achieved.
According to another aspect of the embodiments of the present application, there is also provided a client, where the client is configured to execute the data query method as shown in any one of fig. 2 to fig. 4.
Referring to fig. 5, fig. 5 is a schematic diagram of a client according to an embodiment of the present application.
As shown in fig. 5, the client includes:
the receiving module 11 is configured to receive a data query request input by a user, and receive first fragmentation information sent by a server and aiming at the data query request;
a generating module 12, configured to generate second fragmentation information corresponding to the client according to the data query request and the total number of clients executing the data query request;
an obtaining module 13, configured to obtain a data query result from a preset database according to the first fragmentation information and the second fragmentation information;
and the output module 14 is used for outputting the data query result.
In some embodiments, the data query request carries data information to be queried, and the generating module is configured to generate a data query task according to the data information to be queried, perform fragmentation processing on the data query task according to the total amount, and generate second fragmentation information.
In some embodiments, the generating module 12 is configured to perform fragmentation processing on the data query task according to a preset fragmentation policy, the field value, and the total number, and generate the second fragmentation information;
wherein the fragmentation strategy comprises: at least one of a random fragmentation method, an average fragmentation method, and a remainder fragmentation method.
In some embodiments, the fragmentation policy is the same as a fragmentation policy generated by the server to generate the first fragmentation information according to the data query request.
In some embodiments, if the fragmentation policy is the remainder fragmentation method, the generating module 12 is configured to perform remainder processing on the field value according to the total number to generate the second fragmentation information.
In some embodiments, the obtaining module 13 is configured to determine common fragmentation information of the first fragmentation information and the second fragmentation information, and obtain the data query result from the database according to the common fragmentation information.
In some embodiments, the first sharding information includes a set of first sharding items, the second sharding information includes a set of second sharding items, and the obtaining module 13 is configured to determine an intersection of the set of first sharding items and the set of second sharding items, and obtain the data query result from the database according to the sharding items in the intersection.
According to another aspect of the embodiments of the present application, there is also provided a data query system, including:
the server is used for generating and sending first fragmentation information to the client according to the received data query request:
a client as described in any of the above embodiments, such as the client shown in fig. 5.
According to another aspect of the embodiments of the present application, there is also provided an electronic device, including: a memory, a processor;
a memory for storing processor-executable instructions;
wherein, when executing the instructions in the memory, the processor is configured to implement the method according to any of the above embodiments, such as implementing the data query method according to any of the embodiments shown in fig. 2 to 4.
Referring to fig. 6, fig. 6 is a block diagram of an electronic device according to an embodiment of the present application.
As shown in FIG. 6, the electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of embodiments of the present application described and/or claimed herein.
As shown in fig. 6, the electronic apparatus includes: one or more processors 101, memory 102, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 6, one processor 101 is taken as an example.
The memory 102 is a non-transitory computer readable storage medium provided by the embodiments of the present application. The memory stores instructions executable by at least one processor, so that the at least one processor executes the data query method provided by the embodiment of the application. The non-transitory computer readable storage medium of the embodiments of the present application stores computer instructions for causing a computer to execute the data query method provided by the embodiments of the present application.
The memory 102 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the electronic device, and the like. Further, the memory 102 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 102 may optionally include memory located remotely from processor 101, which may be connected to an electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, Block-chain-Based Service Networks (BSNs), mobile communication networks, and combinations thereof.
The electronic device may further include: an input device 103 and an output device 104. The processor 101, the memory 102, the input device 103, and the output device 104 may be connected by a bus or other means, and fig. 6 illustrates an example of a connection by a bus.
The input device 103 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic apparatus, such as a touch screen, keypad, mouse, track pad, touch pad, pointer stick, one or more mouse buttons, track ball, joystick, or other input device. The output devices 104 may include a display device, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Block-chain-Based Service Networks (BSNs), Wide Area Networks (WANs), and the internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an ASIC. Additionally, the ASIC may reside in user equipment. Of course, the processor and the storage medium may reside as discrete components in a communication device.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (12)
1. A data query method is applied to a client side, and comprises the following steps:
receiving a data query request input by a user, and receiving first fragmentation information aiming at the data query request sent by a server;
generating second fragmentation information corresponding to the clients according to the data query request and the total number of the clients executing the data query request;
and acquiring and outputting a data query result from a preset database according to the first fragmentation information and the second fragmentation information.
2. The method of claim 1, wherein the data query request carries data information to be queried, and the generating of the second fragmentation information according to the data query request and a total number of clients executing the data query request comprises:
generating a data query task according to the data information to be queried;
and carrying out fragmentation processing on the data query data task according to the total number to generate second fragmentation information.
3. The method of claim 2, wherein the data query task comprises a field value, and the generating the data query task according to the data information to be queried comprises:
determining a field name corresponding to the data query request according to the data information to be queried;
generating the field value corresponding to the field name.
4. The method of claim 3, wherein the fragmenting the data query data task according to the total number and generating second fragmentation information comprises:
carrying out fragmentation processing on the data query task according to a preset fragmentation strategy, the field value and the total number to generate second fragmentation information;
wherein the fragmentation strategy comprises: at least one of a random fragmentation method, an average fragmentation method, and a remainder fragmentation method.
5. The method of claim 4, wherein the fragmentation policy is the same as a fragmentation policy generated by the server for generating the first fragmentation information according to the data query request.
6. The method of claim 4, wherein if the fragmentation policy is the remainder fragmentation, performing fragmentation processing on the data query task according to the remainder fragmentation, the field value, and the total number, and generating the second fragmentation information comprises:
and performing remainder processing on the field value according to the total number to generate the second fragmentation information.
7. The method according to any one of claims 1 to 6, wherein the obtaining and outputting a data query result from a preset database according to the first fragmentation information and the second fragmentation information comprises:
determining common fragmentation information of the first fragmentation information and the second fragmentation information;
and acquiring and outputting the data query result from the database according to the common fragmentation information.
8. The method of claim 7, wherein the first fragmentation information comprises a set of first fragmentation entries, wherein the second fragmentation information comprises a set of second fragmentation entries, and wherein the determining fragmentation information common to the first fragmentation information and the second fragmentation information comprises:
determining an intersection of the set of first sharded items and the set of second sharded items;
and the obtaining and outputting the data query result from the database according to the common fragmentation information comprises: and acquiring and outputting the data query result from the database according to the fragment items in the intersection.
9. A client, the client comprising:
the receiving module is used for receiving a data query request input by a user and receiving first fragmentation information aiming at the data query request sent by a server;
the generating module is used for generating second fragmentation information corresponding to the clients according to the data query requests and the total number of the clients executing the data query requests;
the acquisition module is used for acquiring a data query result from a preset database according to the first fragmentation information and the second fragmentation information;
and the output module is used for outputting the data query result.
10. A data query system, the system comprising:
the server is used for generating and sending first fragmentation information to the client according to the received data query request:
the client of claim 9.
11. An electronic device, comprising: a memory, a processor;
a memory for storing the processor-executable instructions;
wherein the processor, when executing the instructions in the memory, is configured to implement the method of any of claims 1 to 8.
12. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011018899.1A CN113760968A (en) | 2020-09-24 | 2020-09-24 | Data query method, device, system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011018899.1A CN113760968A (en) | 2020-09-24 | 2020-09-24 | Data query method, device, system, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113760968A true CN113760968A (en) | 2021-12-07 |
Family
ID=78785853
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011018899.1A Pending CN113760968A (en) | 2020-09-24 | 2020-09-24 | Data query method, device, system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113760968A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116450657A (en) * | 2023-06-19 | 2023-07-18 | 北京好心情互联网医院有限公司 | Data fragment scheduling method, device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170075965A1 (en) * | 2015-09-16 | 2017-03-16 | Turn Inc. | Table level distributed database system for big data storage and query |
| WO2017201970A1 (en) * | 2016-05-21 | 2017-11-30 | 乐视控股（北京）有限公司 | Branch base database system and routing method therefor |
| WO2019141134A1 (en) * | 2018-01-19 | 2019-07-25 | 阿里巴巴集团控股有限公司 | Data query method, apparatus and device |
| CN111338778A (en) * | 2020-02-27 | 2020-06-26 | 苏宁云计算有限公司 | Task scheduling method and device, storage medium and computer equipment |
| CN111597033A (en) * | 2019-02-20 | 2020-08-28 | 北京京东尚科信息技术有限公司 | Task scheduling method and device |
-
2020
- 2020-09-24 CN CN202011018899.1A patent/CN113760968A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170075965A1 (en) * | 2015-09-16 | 2017-03-16 | Turn Inc. | Table level distributed database system for big data storage and query |
| WO2017201970A1 (en) * | 2016-05-21 | 2017-11-30 | 乐视控股（北京）有限公司 | Branch base database system and routing method therefor |
| WO2019141134A1 (en) * | 2018-01-19 | 2019-07-25 | 阿里巴巴集团控股有限公司 | Data query method, apparatus and device |
| CN111597033A (en) * | 2019-02-20 | 2020-08-28 | 北京京东尚科信息技术有限公司 | Task scheduling method and device |
| CN111338778A (en) * | 2020-02-27 | 2020-06-26 | 苏宁云计算有限公司 | Task scheduling method and device, storage medium and computer equipment |
Non-Patent Citations (1)
| Title |
|---|
| 江颉;傅超仪;: "基于查询分片用户协作的位置隐私保护方法", 小型微型计算机系统, no. 05, 14 May 2019 (2019-05-14) * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116450657A (en) * | 2023-06-19 | 2023-07-18 | 北京好心情互联网医院有限公司 | Data fragment scheduling method, device, equipment and storage medium |
| CN116450657B (en) * | 2023-06-19 | 2023-08-29 | 北京好心情互联网医院有限公司 | Data fragment scheduling method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110806923B (en) | Parallel processing method and device for block chain tasks, electronic equipment and medium | |
| JP7214786B2 (en) | Scheduling method, device, device and medium for deep learning inference engine | |
| CN108287764A (en) | Distributed task dispatching method and its system, storage medium, electronic equipment | |
| KR102546060B1 (en) | Blockchain network transaction processing method, apparatus, device, and storage medium | |
| CN111694646A (en) | Resource scheduling method and device, electronic equipment and computer readable storage medium | |
| CN111694857B (en) | Method, device, electronic equipment and computer readable medium for storing resource data | |
| CN111240836A (en) | Computing resource management method and device, electronic equipment and storage medium | |
| CN112100481A (en) | Search method, search device, electronic device, and storage medium | |
| US20160019089A1 (en) | Method and system for scheduling computing | |
| CN111782365A (en) | Timed task processing method, device, equipment and storage medium | |
| CN111880914A (en) | Resource scheduling method, resource scheduling apparatus, electronic device, and storage medium | |
| CN111737399A (en) | Method and device for expanding question and answer set, electronic equipment and readable storage medium | |
| CN110688229B (en) | Task processing method and device | |
| CN112084395A (en) | Search method, search device, electronic device, and storage medium | |
| US20120005580A1 (en) | Autonomic I/O Tracing and Performance Tuning | |
| CN110545324A (en) | Data processing method, device, system, network equipment and storage medium | |
| CN113760968A (en) | Data query method, device, system, electronic equipment and storage medium | |
| CN112579897B (en) | Information searching method and device | |
| CN111881328B (en) | Information pushing method and device, electronic equipment and storage medium | |
| CN113419865A (en) | Cloud resource processing method, related device and computer program product | |
| CN111598571A (en) | Transaction processing method, device, equipment and storage medium of block chain | |
| CN110517079B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN111858030A (en) | Job resource processing method and device, electronic equipment and readable storage medium | |
| CN112069137A (en) | Method and device for generating information, electronic equipment and computer readable storage medium | |
| CN111176838A (en) | Method and device for distributing embedded vectors for nodes in bipartite graph |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |