CN115374340A - Data query method, device, equipment and storage medium - Google Patents

Data query method, device, equipment and storage medium Download PDF

Info

Publication number
CN115374340A
CN115374340A CN202211073659.0A CN202211073659A CN115374340A CN 115374340 A CN115374340 A CN 115374340A CN 202211073659 A CN202211073659 A CN 202211073659A CN 115374340 A CN115374340 A CN 115374340A
Authority
CN
China
Prior art keywords
thread
data
data query
query
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211073659.0A
Other languages
Chinese (zh)
Inventor
柴铭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Netease Hangzhou Network Co Ltd
Original Assignee
Netease Hangzhou Network Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Netease Hangzhou Network Co Ltd filed Critical Netease Hangzhou Network Co Ltd
Priority to CN202211073659.0A priority Critical patent/CN115374340A/en
Publication of CN115374340A publication Critical patent/CN115374340A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/953Querying, e.g. by the use of web search engines
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system

Abstract

The application provides a data query method, a data query device, data query equipment and a storage medium, and relates to the technical field of data query. The method comprises the following steps: receiving a data query request sent by front-end equipment; determining a plurality of data query tasks and a data query task corresponding to each thread in a plurality of threads according to the data query request; executing corresponding data query tasks on the data logs of the preset application respectively by adopting the multiple threads; if a first target thread with overtime execution exists in the multiple threads, monitoring first target data obtained by executing a corresponding data query task on the first target thread by adopting a first preset thread; and returning second target data obtained by the corresponding data query task executed by the second target thread in the multiple threads to the front-end equipment. Compared with the prior art, the problem that data received by the front-end equipment is incomplete is avoided.

Description

Data query method, device, equipment and storage medium
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data query method, apparatus, device, and storage medium.
Background
In the development of a large-scale business system, mass data query conditions of hundreds of millions and billions are often encountered, and due to the large data volume, the response speed of a business interface may be slowed down, so that the experience of a user is affected, and therefore, the response efficiency of the interface needs to be optimized. Due to the complexity of the business, a plurality of different database queries may be involved in one interface, in this case, the query efficiency of the interface generally depends on the query statement with the slowest internal response, and if the query is extremely slow, the response speed of the interface is slowed, so that the user operation experience of the front-end page is finally affected.
In order to solve the above problems, the prior art proposes a query scheme based on multiple threads. The scheme starts a plurality of threads to execute data query logic in the interface, the system distributes different query logic to the threads to execute according to a certain algorithm, and the final execution time of the interface depends on the slowest thread. However, when the thread with the slowest execution speed is stuck, the interface sets a longest waiting time in order to not allow the response of the request initiated by the front end to be overtime, and when the time limit is reached, the interface does not wait for the thread any more, but directly returns the acquired query result to the front end.
However, in such a query method, if there is a timeout of execution of a certain thread, the thread which is executing the timeout will not be waited for, and the data received by the front end is not complete.
Disclosure of Invention
An object of the present application is to provide a method, an apparatus, a device and a storage medium for data query, so as to solve the problem of incomplete data received by a front-end device in the prior art.
In order to achieve the above purpose, the technical solutions adopted in the embodiments of the present application are as follows:
in a first aspect, an embodiment of the present application provides a data query method, where the method includes:
receiving a data query request sent by front-end equipment;
determining a plurality of data query tasks and a data query task corresponding to each thread in a plurality of threads according to the data query request;
executing corresponding data query tasks on the data logs of the preset application respectively by adopting the multiple threads;
if a first target thread with overtime execution exists in the multiple threads, monitoring first target data obtained by executing a corresponding data query task on the first target thread by adopting a first preset thread;
and returning second target data obtained by the corresponding data query task executed by the second target thread in the multiple threads to the front-end equipment.
Optionally, before the monitoring, by using the first preset thread, the first target data obtained by executing the corresponding data query task on the first target thread, the method further includes:
and transmitting the task class of the first target thread to the first preset thread.
Optionally, the method further comprises:
if the first preset thread monitors that the execution of the first target thread is finished, the first preset thread is adopted to store the monitored first target data into a preset database, so that the front-end equipment polls the preset database by adopting a second preset thread to acquire the first target data.
Optionally, if the number of the first target threads is multiple, the storing, by using the first preset thread, the monitored first target data in a preset database includes:
and sequencing a plurality of first target data by adopting the first preset thread, and storing the first target data into the preset database, wherein each first target data is obtained by executing a corresponding data query task by one first target thread.
Optionally, the data query request includes: the method comprises the following steps of (1) inquiring a range, and analyzing units and index items corresponding to the inquiring range; the determining a plurality of data query tasks and a data query task corresponding to each thread in a plurality of threads according to the data query request includes:
splitting the query range according to the analysis unit to obtain a plurality of sub-query ranges;
respectively generating the plurality of data query tasks according to the plurality of sub-query ranges and the index items;
distributing the plurality of data query tasks to the plurality of threads.
Optionally, if the query range is: a time range, the analysis unit being a unit of time; splitting the query range according to the analysis unit to obtain a plurality of sub-query ranges, including:
and splitting the time range according to the unit time to obtain a plurality of time intervals, wherein the plurality of sub-query ranges are the plurality of time intervals.
Optionally, if the query range is: a data dimension range, wherein the analysis unit is a unit dimension; splitting the query range according to the analysis unit to obtain a plurality of sub-query ranges, including:
and splitting the data dimension range according to the unit dimension to obtain a plurality of sub-dimensions, wherein the plurality of sub-query ranges are the plurality of sub-dimensions.
Optionally, the data dimension range is: a range of any one of an application scene dimension, a platform dimension, a region dimension, and a channel dimension.
Optionally, before the monitoring, by using the first preset thread, the first target data obtained by executing the corresponding data query task on the first target thread, the method further includes:
detecting whether the execution timeout exists in the multiple threads by adopting a detector started in advance;
and if the first target thread with overtime execution exists in the multiple threads, starting the first preset thread.
In a second aspect, another embodiment of the present application provides a data query apparatus, including: the device comprises a receiving module, a determining module, a query module, a monitoring module and a returning module, wherein:
the receiving module is used for receiving a data query request sent by the front-end equipment;
the determining module is used for determining a plurality of data query tasks and a data query task corresponding to each thread in a plurality of threads according to the data query request;
the query module is used for executing corresponding data query tasks on the data logs of the preset application by adopting the multiple threads;
the monitoring module is used for monitoring first target data obtained by executing a corresponding data query task on a first target thread by adopting a first preset thread if the first target thread with overtime execution exists in the plurality of threads;
and the return module is used for returning second target data obtained by the corresponding data query task executed by the second target thread which is executed in the multiple threads to the front-end equipment.
Optionally, the apparatus further comprises: and the sending module is used for transmitting the task class of the first target thread to the first preset thread.
Optionally, the apparatus further comprises: and the storage module is used for storing the monitored first target data into a preset database by adopting the first preset thread if the first preset thread monitors that the execution of the first target thread is finished, so that the front-end equipment polls the preset database by adopting a second preset thread to acquire the first target data.
Optionally, the storage module is specifically configured to employ the first preset thread, sequence a plurality of first target data, and store the first target data in the preset database, where each first target data is data obtained by executing a corresponding data query task by one first target thread.
Optionally, the apparatus further comprises: split module, generation module and distribution module, wherein:
the splitting module is used for splitting the query range according to the analysis unit to obtain a plurality of sub-query ranges;
the generating module is used for respectively generating the plurality of data query tasks according to the plurality of sub-query ranges and the index items;
the distribution module is used for distributing the data query tasks to the threads.
Optionally, if the query range is: a time range, the analysis unit being a unit of time; the splitting module is specifically configured to split the time range according to the unit time to obtain a plurality of time intervals, where the plurality of sub-query ranges are the plurality of time intervals.
Optionally, if the query range is: a data dimension range, wherein the analysis unit is a unit dimension; the splitting module is specifically configured to split the data dimension range according to the unit dimension to obtain a plurality of sub-dimensions, where the plurality of sub-query ranges are the plurality of sub-dimensions.
Optionally, the data dimension range is: a range of any one of an application scene dimension, a platform dimension, a region dimension, and a channel dimension.
Optionally, the apparatus further comprises: detection module and opening module, wherein:
the detection module is used for detecting whether the execution timeout of the multiple threads exists by adopting a detector started in advance;
the starting module is configured to start the first preset thread if the first target thread with an execution timeout exists in the multiple threads.
In a third aspect, another embodiment of the present application provides a data query device, including: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the data query device is operated, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the method according to any one of the first aspect.
In a fourth aspect, another embodiment of the present application provides a storage medium having a computer program stored thereon, where the computer program is executed by a processor to perform the steps of the method according to any one of the above first aspects.
The beneficial effect of this application is: by adopting the data query method provided by the application, after a data query request sent by a front-end device is received, the data query task corresponding to each thread in a plurality of threads can be determined according to the data query request, each corresponding thread executes each data query task on a preset applied data log, a second target thread after execution returns second target data obtained by executing the corresponding data query task to the front-end device, if a first target thread with overtime execution exists in the execution process of each task, the first target data obtained by executing the corresponding data query task on the first target thread is monitored by adopting the first preset thread, so that the first preset thread can continuously monitor the first target thread even if the first target thread with overtime execution exists, the execution result of the first preset thread can be received by adopting the first preset thread, and therefore, by adopting the method provided by the application, the query rate is improved by a plurality of threads in a parallel query mode, and the integrity of the plurality of data can be obtained by monitoring the execution result of the plurality of threads even if the first target thread with overtime execution exists in the plurality of threads.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a schematic flowchart of a data query method according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a data query method according to another embodiment of the present application;
fig. 3 is a schematic flowchart of a data query method according to another embodiment of the present application;
fig. 4 is a schematic structural diagram of a data query apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a data query device according to another embodiment of the present application;
fig. 6 is a schematic structural diagram of a data query device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments.
The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
Additionally, the flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and that steps without logical context may be reversed in order or performed concurrently. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.
A data query method provided in the embodiments of the present application is explained below with reference to a plurality of specific application examples. Fig. 1 is a schematic flowchart of a data query method according to an embodiment of the present application, and as shown in fig. 1, the method includes:
s101: and receiving a data query request sent by the front-end equipment.
In an embodiment of the present application, the data query request may include, for example: the query range, the analysis units corresponding to the query range and the index items; the data query request may be, for example, a query dimension (i.e., a query range) to be queried and a query indicator (i.e., an indicator item) in the query dimension, which are selected or input by a user on a display interface of the front-end device, and then the user determines an aggregation unit (i.e., an analysis unit).
For example, if the query range is: time range, the analysis unit may be, for example, unit time, for example, when the query range is No. 1 to No. 7 of a month, the analysis unit may be, for example, every day (i.e., 24 hours), then seven analysis intervals will be obtained, and each analysis interval includes one analysis unit (i.e., 24 hours of data); the analysis unit may be 12 hours, in which case fourteen analysis intervals are obtained, each analysis interval including one analysis unit (i.e., 12 hours of data), or the analysis unit may be any aggregation time.
If the query range is: the data dimension range, the analysis unit may be, for example, a unit dimension. In some possible embodiments, the data dimension range may be, for example: a range of any one of an application scene dimension, a platform dimension, a region dimension, and a channel dimension.
The platform dimension may be, for example, a device model dimension of the user terminal, a mobile phone platform dimension of the user terminal, a vendor dimension, or the like; the regional dimensions may be, for example: country dimension, city dimension, or region dimension, province dimension, and the like; it should be understood that the foregoing embodiments are merely exemplary, and the dividing manner of the specific dimension may be adjusted according to the user's needs, and is not limited to the foregoing embodiments.
For example, when the query range is a platform dimension, the analysis unit may be, for example: the number of users, or the number of Frames transmitted Per Second (Frames Per Second, fps), etc.; when the query range is a country dimension, the analysis unit may be, for example: it should be understood that the above embodiments are merely exemplary, and the specific query range may include a range and the analysis unit information corresponding to each query range may be flexibly adjusted according to the user's needs, and is not limited to the above embodiments.
S102: and determining a plurality of data query tasks and a data query task corresponding to each thread in the plurality of threads according to the data query request.
In the embodiment of the present application, the manner of determining the data query task corresponding to each thread may be, for example: splitting the query range according to an analysis unit to obtain a plurality of sub-query ranges; respectively generating a plurality of data query tasks according to the plurality of sub-query ranges and the index items; the plurality of data query tasks are distributed to the plurality of threads.
The system can dynamically allocate the corresponding data query tasks to each thread according to the analysis unit preset by the user, and each thread is responsible for querying data corresponding to one coordinate point in the result graph.
For example, when the query range is a time range and the analysis unit is a unit time, the method for determining the plurality of sub-query ranges is obtained by splitting the time, and may be, for example: splitting the time range according to preset unit time to obtain a plurality of time intervals, wherein the plurality of sub-query ranges are the plurality of time intervals; for example, in one possible embodiment, the time range determined by the user is 7-14 of a month, the analysis unit is 1 day, the system performs segmentation on the time range by taking 24 hours as a unit, and then, according to the segmentation result, 7 threads are started, each thread is used for querying the index condition of each day in the dimension selected by the user; if the analysis unit determined by the user is 12 hours, the system divides the time range by taking 12 hours as a unit and starts 14 threads to execute tasks; by analogy, it can be seen that the user can select any analysis unit, and the system can calculate the number of threads which need to be simultaneously started according to the analysis unit selected by the user.
Still taking the query scope as a time scope and the analysis unit as a time unit for illustration, since the time scope selected by the user may be weeks, months or even longer, and the data that the user wishes to analyze may be in hours, or days or longer. Therefore, the front end needs to transmit the start-stop time (time range), the aggregation time unit (analysis unit) and the selected query condition (index item) selected by the user to the back end interface; however, since the query request received by the backend is a query request, the query request needs to be split into a plurality of data query tasks, and then each split data query task is allocated to each thread; the splitting aims at that one thread is responsible for inquiring data of a coordinate point in a last ten-day result graph manufactured by front-end equipment, when the inquiry range is a time range, the system can generate an empty time list according to the time range and time units, then starting from the starting time, sequentially storing the starting time of each time period into the time list by taking the time units as units, and the splitting of the time range is finished.
In the query scope: the data dimension range, when the analysis unit is a unit dimension, the manner of determining the plurality of sub-query ranges is to split the dimension, and may be, for example: and splitting the data dimension range according to the unit dimension to obtain a plurality of sub-dimensions, wherein the plurality of sub-query ranges are the plurality of sub-dimensions.
S103: and adopting a plurality of threads to respectively execute corresponding data query tasks on the data logs of the preset application.
After receiving the data query tasks corresponding to the threads, the threads start to execute the corresponding data query tasks, and the query range is still taken as an example for explanation, wherein the number of the started threads depends on the number of elements (also called the number of time periods) in the generated time list, and each thread is responsible for data query in one time period in the time list; when the query range is a region range, for example, a country range, the number of the started threads depends on the number of elements (that is, the number of countries) in the generated region list, and each thread is responsible for querying data corresponding to one country in the list; it should be understood that the above embodiments are only illustrative, and the specific implementation process is not limited thereto.
S104: and if a first target thread with overtime execution exists in the multiple threads, monitoring first target data obtained by executing the corresponding data query task on the first target thread by adopting a first preset thread.
In the execution process of each thread, a phenomenon that one or more threads are overtime due to network abnormality, thread failure or other conditions may occur, in order not to affect user experience of a front-end device user, in the embodiment of the present application, the response time of each thread needs to be limited, when the overtime time of the overtime thread is greater than a preset overtime time, the thread is determined to be a first target thread with overtime execution, and the first target thread is monitored by the first preset thread; the preset timeout time can be flexibly adjusted or set according to the needs of users.
S105: and returning second target data obtained by the corresponding data query task executed by the second target thread in the multiple threads to the front-end equipment.
In the embodiment of the present application, after the front end receives the returned second target data, the data query graph continues to be drawn according to the second target data.
When each data query task is determined according to the time dimension, the abscissa of the data query graph is time, the ordinate is a query index (namely an index item), and the data of each coordinate point is obtained by executing the corresponding data query task through a corresponding thread; when determining each query task by using data dimensions, for example, when a user wants to analyze a query index corresponding to each city in a country, a city and the query index corresponding to the city are a coordinate point at the moment, and data of the coordinate point is obtained by executing the corresponding query task by a corresponding thread; the data query graphs corresponding to other dimensions are similar in situation, and are not described herein again.
By adopting the data query method provided by the application, after a data query request sent by a front-end device is received, a data query task corresponding to each thread in a plurality of threads can be determined according to the data query request, each corresponding thread executes each data query task on a preset applied data log, a second target thread after execution is returned to the front-end device, and second target data obtained by executing the corresponding data query task on the second target thread is obtained.
Optionally, on the basis of the above embodiments, the embodiments of the present application may further provide a data query method, and an implementation process of the method is described as follows with reference to the accompanying drawings. Fig. 2 is a schematic flowchart of a data query method according to another embodiment of the present application, as shown in fig. 2, before S104, the method may further include:
s111: and transmitting the task class of the first target thread to a first preset thread.
If the first preset thread monitors that the execution of the first target thread is finished, the first preset thread is adopted to store the monitored first target data into a preset database, so that the front-end equipment polls the preset database by adopting a second preset thread to acquire the first target data.
In an embodiment of the present application, since the front-end device has received second target data obtained by executing a corresponding data query task by another thread that is not executed overtime, and draws a data query graph according to the obtained second target data, even if there are some first target threads that are executed overtime, a front-end user may also see the data query graph drawn by the second target data, an execution time of each first target thread in the first preset threads does not affect a user experience of the front-end, the first preset thread may wait for execution of the first target threads that are executed overtime, and after execution of a plurality of first target threads is completed, use the first preset thread to sort a plurality of first target data, and store the first target data in a preset database, where the preset database may be, for example, a Remote Dictionary service (Remote Dictionary Server, redis), or a relational database management system (MySQL), and the like.
In some possible embodiments, after the first target process sorts the plurality of first target data and stores the sorted first target data in the preset database, the first target process stores 1 in the execution flag bit in the returned field to indicate that the first target process is a process whose execution is completed.
In the implementation of the application, in the process that a user waits for acquiring data of a first target table, the user may perform other operations, for example, the user may query a data query graph drawn according to second target data, and the like, without affecting user experience, after the front end acquires first target data in a preset database by using a second preset thread, the acquired first target data may be supplemented in the data query graph, so far, data on the data query graph is complete data obtained by executing a corresponding data query task on a data log of a preset application, and thus, the front end user is ensured to still acquire complete data even when there is a task of executing an execution timeout, and integrity of the query data is ensured.
Optionally, on the basis of the foregoing embodiments, the embodiments of the present application may further provide a data query method, and an implementation process of the foregoing method is illustrated in the following with reference to the accompanying drawings. Fig. 3 is a schematic flowchart of a data query method according to another embodiment of the present application, and as shown in fig. 3, before S104, the method may further include:
s121: and detecting whether the execution time-out exists in the plurality of threads by adopting a detector which is opened in advance.
In the embodiment of the present application, in the process of executing a data query task corresponding to a data log of a preset application by a plurality of threads, a detector that is turned on in advance needs to be used to check the execution time of each thread to determine whether the execution time of each thread exceeds a preset execution time, and if so, the thread that exceeds the preset execution time is determined to be a first target thread that executes a timeout.
S122: and if a first target thread with overtime execution exists in the multiple threads, starting a first preset thread.
For example, in some possible embodiments, when a plurality of data query tasks and a data query task corresponding to each thread in the threads are determined, a first preset thread is created, where the first preset thread is used to receive any first target thread that performs data query task thread delivery; or, in other possible embodiments, in the process of executing a plurality of data query tasks, if a first target thread with a query timeout occurs for the first time, starting a thread as a first preset thread, and transmitting the first target thread with the execution timeout to the first preset thread, where the first preset thread continues to wait for an execution result of the first target thread; it should be understood that the creation time and the creation manner of the specific first preset thread may be flexibly adjusted according to the user requirement, and are not limited to the embodiments described above.
By adopting the data query method provided by the application, the number of threads needing to be started simultaneously can be calculated by the system according to a preset algorithm, each thread is responsible for executing the data query tasks, then the threads are started simultaneously to execute the data query tasks, compared with single-thread query, the method provided by the application can greatly improve the query efficiency and improve the response speed of the interface.
The following explains a data query apparatus provided in the present application with reference to the drawings, where the data query apparatus can execute any one of the data query methods in fig. 1 to 3, and specific implementation and beneficial effects of the data query apparatus refer to the above description, which is not described again below.
Fig. 4 is a schematic structural diagram of a data query apparatus according to an embodiment of the present application, and as shown in fig. 5, the apparatus includes: a receiving module 201, a determining module 202, a querying module 203, a monitoring module 204, and a returning module 205, wherein:
a receiving module 201, configured to receive a data query request sent by a front-end device;
a determining module 202, configured to determine, according to the data query request, a plurality of data query tasks and a data query task corresponding to each of the plurality of threads;
the query module 203 is configured to employ multiple threads to execute corresponding data query tasks on the preset applied data logs respectively;
the monitoring module 204 is configured to, if a first target thread which is executed overtime exists in the multiple threads, monitor, by using a first preset thread, first target data obtained by executing a corresponding data query task on the first target thread;
the returning module 205 is configured to return second target data obtained by a corresponding data query task executed by a second target thread that is executed in the multiple threads to the front-end device.
Alternatively, on the basis of the above embodiments, the embodiments of the present application may also provide a data query apparatus, and an implementation process of the apparatus shown in fig. 4 is described as follows with reference to the accompanying drawings. Fig. 5 is a schematic structural diagram of a data query apparatus according to another embodiment of the present application, and as shown in fig. 5, the apparatus further includes: the sending module 206 is configured to transfer the task class of the first target thread to a first preset thread.
As shown in fig. 5, the apparatus further includes: the storage module 207 is configured to, if the first preset thread monitors that the execution of the first target thread is completed, store the monitored first target data in a preset database by using the first preset thread, so that the front-end device polls the preset database by using a second preset thread to obtain the first target data.
Optionally, the storage module 207 is specifically configured to employ a first preset thread, sort multiple first target data, and store the sorted multiple first target data in a preset database, where each first target data is data obtained by executing a corresponding data query task by a first target thread.
As shown in fig. 5, the apparatus further includes: a splitting module 208, a generating module 209, and an assigning module 210, wherein:
a splitting module 208, configured to split the query range according to an analysis unit to obtain multiple sub-query ranges;
a generating module 209, configured to generate a plurality of data query tasks according to the plurality of sub-query ranges and the index items;
an assigning module 210 for assigning the plurality of data query tasks to the plurality of threads.
Optionally, if the query range is: time range, the unit of analysis is unit time; the splitting module 208 is specifically configured to split the time range according to the unit time to obtain multiple time intervals, where the multiple sub-query ranges are multiple time intervals.
Optionally, if the query range is: a data dimension range, wherein an analysis unit is a unit dimension; the splitting module 208 is specifically configured to split the data dimension range according to the unit dimension to obtain multiple sub-dimensions, where the multiple sub-query ranges are multiple sub-dimensions.
Optionally, the data dimension range is: a range of any one of an application scene dimension, a platform dimension, a region dimension, and a channel dimension.
As shown in fig. 5, the apparatus further includes: a detection module 211 and an opening module 212, wherein:
a detecting module 211, configured to detect whether execution of multiple threads is overtime by using a pre-turned-on detector;
the starting module 212 is configured to start a first preset thread if a first target thread with an execution timeout exists among the plurality of threads.
The above-mentioned apparatus is used for executing the method provided by the foregoing embodiment, and the implementation principle and technical effect are similar, which are not described herein again.
These above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors, or one or more Field Programmable Gate Arrays (FPGAs), etc. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. As another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Fig. 6 is a schematic structural diagram of a data query device according to an embodiment of the present application, where the data query device may be integrated in a terminal device or a chip of the terminal device.
As shown in fig. 6, the data query apparatus includes: a processor 501, a storage medium 502, and a bus 503.
The processor 501 is used for storing a program, and the processor 501 calls the program stored in the storage medium 502 to execute the method embodiment corresponding to fig. 1-3. The specific implementation and technical effects are similar, and are not described herein again.
Optionally, the present application also provides a program product, such as a storage medium, on which a computer program is stored, including a program, which, when executed by a processor, performs embodiments corresponding to the above-described method.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or in the form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer-readable storage medium. The software functional unit is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (in english: processor) to execute some steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a portable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other media capable of storing program codes.

Claims (12)

1. A method for data query, the method comprising:
receiving a data query request sent by front-end equipment;
determining a plurality of data query tasks and a data query task corresponding to each thread in a plurality of threads according to the data query request;
executing corresponding data query tasks on the data logs of the preset application respectively by adopting the multiple threads;
if a first target thread with overtime execution exists in the multiple threads, monitoring first target data obtained by executing a corresponding data query task on the first target thread by adopting a first preset thread;
and returning second target data obtained by executing the corresponding data query task to the front-end equipment for a second target thread which is executed in the multiple threads.
2. The method of claim 1, wherein before monitoring, with a first predetermined thread, first target data obtained by the first target thread executing a corresponding data query task, the method further comprises:
and transmitting the task class of the first target thread to the first preset thread.
3. The method of claim 2, wherein the method further comprises:
if the first preset thread monitors that the first target thread is executed completely, the first preset thread is adopted to store the monitored first target data into a preset database, so that the front-end equipment adopts a second preset thread to poll the preset database to obtain the first target data.
4. The method of claim 3, wherein if the number of the first target threads is multiple, the storing the monitored first target data in a preset database by using the first preset thread comprises:
and sequencing a plurality of first target data by adopting the first preset thread, and storing the first target data into the preset database, wherein each first target data is obtained by executing a corresponding data query task by one first target thread.
5. The method of claim 1, wherein the data query request comprises: the method comprises the following steps of (1) inquiring a range, and analyzing units and index items corresponding to the inquiring range; the determining a plurality of data query tasks and a data query task corresponding to each thread in a plurality of threads according to the data query request includes:
splitting the query range according to the analysis unit to obtain a plurality of sub-query ranges;
respectively generating the plurality of data query tasks according to the plurality of sub-query ranges and the index items;
distributing the plurality of data query tasks to the plurality of threads.
6. The method of claim 5, wherein if the query range is: a time range, the unit of analysis being a unit of time; splitting the query range according to the analysis unit to obtain a plurality of sub-query ranges, including:
and splitting the time range according to the unit time to obtain a plurality of time intervals, wherein the plurality of sub-query ranges are the plurality of time intervals.
7. The method of claim 5, wherein if the query range is: a data dimension range, wherein the analysis unit is a unit dimension; splitting the query range according to the analysis unit to obtain a plurality of sub-query ranges, including:
and splitting the data dimension range according to the unit dimension to obtain a plurality of sub-dimensions, wherein the plurality of sub-query ranges are the plurality of sub-dimensions.
8. The method of claim 7, wherein the range of data dimensions is: a range of any one of an application scene dimension, a platform dimension, a region dimension, and a channel dimension.
9. The method of claim 1, wherein before monitoring, with a first predetermined thread, first target data obtained by the first target thread executing a corresponding data query task, the method further comprises:
detecting whether the execution timeout exists in the multiple threads by adopting a detector started in advance;
and if the first target thread with overtime execution exists in the multiple threads, starting the first preset thread.
10. A data query apparatus, characterized in that the apparatus comprises: the device comprises a receiving module, a determining module, an inquiring module, a monitoring module and a returning module, wherein:
the receiving module is used for receiving a data query request sent by the front-end equipment;
the determining module is used for determining a plurality of data query tasks and a data query task corresponding to each thread in a plurality of threads according to the data query request;
the query module is used for executing corresponding data query tasks on the data logs of the preset application by adopting the multiple threads;
the monitoring module is used for monitoring first target data obtained by executing a corresponding data query task on a first target thread by adopting a first preset thread if the first target thread with overtime execution exists in the plurality of threads;
and the return module is used for returning second target data obtained by the corresponding data query task executed by the second target thread which is executed in the multiple threads to the front-end equipment.
11. A data query device, the device comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating via the bus when the data query device is running, the processor executing the machine-readable instructions to perform the method of any one of claims 1-9.
12. A storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, performs the method of any of the preceding claims 1-9.
CN202211073659.0A 2022-09-02 2022-09-02 Data query method, device, equipment and storage medium Pending CN115374340A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211073659.0A CN115374340A (en) 2022-09-02 2022-09-02 Data query method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211073659.0A CN115374340A (en) 2022-09-02 2022-09-02 Data query method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115374340A true CN115374340A (en) 2022-11-22

Family

ID=84070284

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211073659.0A Pending CN115374340A (en) 2022-09-02 2022-09-02 Data query method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115374340A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117171102A (en) * 2023-09-07 2023-12-05 山东九州信泰信息科技股份有限公司 Method for writing files at high speed in multithreading and lock-free mode

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117171102A (en) * 2023-09-07 2023-12-05 山东九州信泰信息科技股份有限公司 Method for writing files at high speed in multithreading and lock-free mode
CN117171102B (en) * 2023-09-07 2024-01-26 山东九州信泰信息科技股份有限公司 Method for writing files at high speed in multithreading and lock-free mode

Similar Documents

Publication Publication Date Title
US11379475B2 (en) Analyzing tags associated with high-latency and error spans for instrumented software
CN108009236B (en) Big data query method, system, computer and storage medium
CN106649670B (en) Data monitoring method and device based on stream computing
US11775501B2 (en) Trace and span sampling and analysis for instrumented software
AU2012236577B2 (en) Processing data in a mapreduce framework
CN104125163B (en) Data processing method and device and terminal
EP2738665A1 (en) Similarity analysis method, apparatus, and system
WO2019153487A1 (en) System performance measurement method and device, storage medium and server
CN107861981B (en) Data processing method and device
CN109783512A (en) Data processing method, device, computer equipment and storage medium
US20190042621A1 (en) Query Method and Query Device
CN112328688B (en) Data storage method, device, computer equipment and storage medium
CN115374340A (en) Data query method, device, equipment and storage medium
CN111381940B (en) Distributed data processing method and device
CN111694793A (en) Log storage method and device and log query method and device
CN112445833A (en) Data paging query method, device and system for distributed database
CN114398520A (en) Data retrieval method, system, device, electronic equipment and storage medium
CN112182043A (en) Log data query method, device, equipment and storage medium
CN107193749B (en) Test method, device and equipment
CN113987086A (en) Data processing method, data processing device, electronic device, and storage medium
CN112612832B (en) Node analysis method, device, equipment and storage medium
CN110764705B (en) Data reading and writing method, device, equipment and storage medium
CN109101595B (en) Information query method, device, equipment and computer readable storage medium
CN111159229B (en) Data query method and device
CN113448985A (en) API (application program interface) interface generation method, calling method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination