CN112445814A

CN112445814A - Data acquisition method and device, computer equipment and storage medium

Info

Publication number: CN112445814A
Application number: CN202011479351.7A
Authority: CN
Inventors: 王晨; 叶子奇; 王凡; 张晓龙
Original assignee: Beijing Lexuebang Network Technology Co ltd
Current assignee: Beijing Lexuebang Network Technology Co ltd
Priority date: 2020-12-15
Filing date: 2020-12-15
Publication date: 2021-03-05

Abstract

The present disclosure provides a data acquisition method, apparatus, computer device and storage medium, the method comprising: receiving a data acquisition request, and acquiring at least one screening dimension corresponding to the data acquisition request; matching at least one screening dimension corresponding to the data acquisition request with at least one pre-polymerization dimension combination; the prepolymerization dimension combination corresponds to prepolymerization data obtained by polymerization treatment according to the prepolymerization dimension combination; and using the pre-polymerization data corresponding to the successfully matched target pre-polymerization dimension combination as the target data corresponding to the data acquisition request. Therefore, the difficulty in data screening and aggregation caused by excessive data dimensionality and excessive data volume can be avoided, the time for acquiring target data is shortened, and the data acquisition efficiency is improved.

Description

Data acquisition method and device, computer equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a data acquisition method and apparatus, a computer device, and a storage medium.

Background

In the current information explosion age, along with the rapid development of computers and network technologies, an increasingly huge amount of data appears. Data is used as a carrier for carrying information in a computer, the dimension of the data is not single, and the complexity of the data grows exponentially with the increase of the dimension of the data. In the related art, when data is acquired, generally, after a data acquisition request is received, required data is screened from mass data stored in a database according to required data dimensions in the data acquisition request, the screened data is subjected to aggregation processing, and then the aggregated data is used as data corresponding to the data request.

However, in this process, on one hand, because the amount of data stored in the database is large, time consumed for searching for data from the database directly according to the data dimension required by the data acquisition request may be large, and on the other hand, if the dimension corresponding to the screened data is too large, time required for aggregation processing may be large in the process of data aggregation, and thus the data acquisition method is inefficient.

Disclosure of Invention

The embodiment of the disclosure at least provides a data acquisition method, a data acquisition device, computer equipment and a storage medium.

In a first aspect, an embodiment of the present disclosure provides a data obtaining method, including:

receiving a data acquisition request, and acquiring at least one screening dimension corresponding to the data acquisition request;

matching at least one screening dimension corresponding to the data acquisition request with at least one pre-polymerization dimension combination; the prepolymerization dimension combination corresponds to prepolymerization data obtained by polymerization treatment according to the prepolymerization dimension combination; and using the pre-polymerization data corresponding to the successfully matched target pre-polymerization dimension combination as the target data corresponding to the data acquisition request.

In a possible embodiment, at least one pre-polymerization dimensional combination is determined according to the following method:

in response to a selection instruction for at least one second target dimension of a plurality of first target dimensions, at least one pre-polymerization dimension combination is determined based on the plurality of first target dimensions and the at least one second target dimension.

arranging and combining the plurality of screening dimensions to generate a plurality of dimension combinations to be screened;

and determining a pre-polymerization dimension combination from the dimension combinations to be screened based on the calling frequency and/or the historical calling times of the dimension combinations to be screened.

In a possible embodiment, the determining at least one pre-polymerization dimension combination based on the plurality of first target dimensions and the at least one second target dimension comprises:

determining a third target dimension of the plurality of first target dimensions other than the at least one second target dimension based on the at least one second target dimension;

determining the at least one pre-polymerization dimension combination based on the third target dimension.

In a possible embodiment, the method further comprises:

after matching at least one screening dimension corresponding to the data acquisition request with the at least one pre-polymerization dimension combination, if the matching fails, determining a data attribute corresponding to the data acquisition request, wherein the data attribute is used for indicating whether data requested to be acquired by the data request can be polymerized;

if the data attribute is polymerizable, performing polymerization processing on pre-polymerization data corresponding to the at least one pre-polymerization dimension combination based on the at least one pre-polymerization dimension combination and the at least one screening dimension to obtain target data corresponding to the data acquisition request.

In one possible embodiment, the pre-polymerization data comprises at least one data;

the aggregating the pre-polymerization data corresponding to the at least one pre-polymerization dimension combination to obtain the target data corresponding to the data acquisition request includes:

determining an aggregation mode corresponding to the data acquisition request;

and according to the corresponding aggregation mode, performing aggregation processing on the pre-aggregation data corresponding to the at least one pre-aggregation dimension combination to obtain target data corresponding to the data acquisition request.

In one possible embodiment, the polymerization mode comprises at least one of the following modes:

summing, calculating difference, calculating maximum value, calculating minimum value, calculating intersection and calculating union.

In a possible implementation manner, if the data attribute corresponding to the data obtaining request is non-aggregatable, the method further includes:

and performing data screening based on at least one screening dimension corresponding to the data acquisition request to obtain target data corresponding to the data request.

In a possible implementation manner, after performing data screening based on at least one screening dimension corresponding to the data obtaining request to obtain target data corresponding to the data request, the method further includes:

and taking at least one screening dimension corresponding to the data acquisition request as a pre-polymerization dimension combination, taking the obtained target data as pre-polymerization data corresponding to the pre-polymerization dimension combination, and storing the pre-polymerization data.

In a possible embodiment, the method further comprises:

determining a calling frequency corresponding to the at least one pre-polymerization dimension combination;

and deleting the corresponding pre-polymerization dimension combination with the calling frequency less than the preset frequency and the pre-polymerization data corresponding to the pre-polymerization dimension combination.

In a second aspect, an embodiment of the present disclosure further provides a data obtaining apparatus, including:

the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for receiving a data acquisition request and acquiring at least one screening dimension corresponding to the data acquisition request;

the matching module is used for matching at least one screening dimension corresponding to the data acquisition request with at least one pre-polymerization dimension combination; the prepolymerization dimension combination corresponds to prepolymerization data obtained by polymerization treatment according to the prepolymerization dimension combination; and using the pre-polymerization data corresponding to the successfully matched target pre-polymerization dimension combination as the target data corresponding to the data acquisition request.

In a possible embodiment, the matching module is configured to determine at least one pre-polymerization dimensional combination according to the following method:

In a possible embodiment, the matching module, when determining at least one pre-polymerization dimension combination based on the plurality of first target dimensions and the at least one second target dimension, is configured to:

In a possible implementation, the matching module is further configured to:

the matching module, when performing aggregation processing on the pre-polymerization data corresponding to the at least one pre-polymerization dimension combination to obtain target data corresponding to the data acquisition request, is configured to:

determining an aggregation mode corresponding to the data acquisition request;

In a possible implementation manner, if the data attribute corresponding to the data obtaining request is non-aggregatable, the matching module is further configured to:

In a possible implementation manner, after performing data screening based on at least one screening dimension corresponding to the data obtaining request to obtain target data corresponding to the data request, the matching module is further configured to:

In a possible implementation, the matching module is further configured to:

In a third aspect, an embodiment of the present disclosure further provides a computer device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the computer device is running, the machine-readable instructions when executed by the processor performing the steps of the first aspect described above, or any possible implementation of the first aspect.

In a fourth aspect, this disclosed embodiment also provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps in the first aspect or any one of the possible implementation manners of the first aspect.

According to the data acquisition method, the data acquisition device, the computer equipment and the storage medium provided by the embodiment of the disclosure, after a data acquisition request is received, at least one screening dimension corresponding to the data acquisition request can be matched with at least one pre-polymerization dimension combination, and the pre-polymerization dimension combination corresponds to pre-polymerization data, so that after the matching is successful, the pre-polymerization data corresponding to the pre-polymerization dimension combination can be directly used as target data corresponding to the data acquisition request, thereby avoiding data screening and aggregation difficulties caused by excessive data dimensions and excessive data quantity, reducing the time for acquiring the target data, and rapidly matching the corresponding target data for the data acquisition request after the data acquisition request is received, thereby improving the efficiency of data acquisition.

In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.

Fig. 1 shows a flowchart of a data acquisition method provided by an embodiment of the present disclosure;

fig. 2 is a flowchart illustrating a specific method for determining a pre-polymerization dimension combination in a data acquisition method provided in an embodiment of the present disclosure;

fig. 3 shows a setting page for determining the first target dimension and the second target dimension in the data acquisition method provided by the embodiment of the present disclosure;

FIG. 4 shows a data screening interface in the data acquisition method provided by the embodiment of the present disclosure;

fig. 5 is a flowchart illustrating a specific method for obtaining the target data in the data obtaining method according to the embodiment of the present disclosure;

fig. 6 is a flowchart illustrating a specific method for deleting a pre-polymerization dimension combination in a data acquisition method provided in an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of a data acquisition device provided by an embodiment of the present disclosure;

fig. 8 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

The term "and/or" herein merely describes an associative relationship, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

Research shows that the complexity of the data grows exponentially with the increase of the data dimension, so that the time for acquiring the data from the database is greatly increased. If data is screened for one time, and data is acquired from the database storing mass data according to the screening dimension established by the data screening for the time, the acquisition efficiency is low.

Based on the above research, the present disclosure provides a data acquisition method, after a data acquisition request is received, at least one screening dimension corresponding to the data acquisition request may be matched with at least one pre-polymerization dimension combination, and since the pre-polymerization dimension combination corresponds to pre-polymerization data, after matching is successful, the pre-polymerization data corresponding to the pre-polymerization dimension combination may be directly used as target data corresponding to the data acquisition request, so that difficulties in data screening and aggregation caused by excessive data dimensions and excessive data volume may be avoided, time for acquiring the target data may be reduced, after the data acquisition request is received, corresponding target data may be quickly matched for the data acquisition request, and efficiency of data acquisition may be improved.

To facilitate understanding of the present embodiment, first, a data acquisition method disclosed in an embodiment of the present disclosure is described in detail, where an execution subject of the data acquisition method provided in the embodiment of the present disclosure is generally a computer device with certain computing capability, and the computer device includes, for example: a terminal device, which may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle mounted device, a wearable device, or a server or other processing device. In some possible implementations, the data acquisition method may be implemented by a processor calling computer readable instructions stored in a memory.

Referring to fig. 1, a flowchart of a data acquisition method provided in the embodiment of the present disclosure is shown, where the method includes steps S101 to S102, where:

s101: receiving a data acquisition request, and acquiring at least one screening dimension corresponding to the data acquisition request.

S102: matching at least one screening dimension corresponding to the data acquisition request with at least one pre-polymerization dimension combination; the prepolymerization dimension combination corresponds to prepolymerization data obtained by polymerization treatment according to the prepolymerization dimension combination; and using the pre-polymerization data corresponding to the successfully matched target pre-polymerization dimension combination as the target data corresponding to the data acquisition request.

Each step and the corresponding implementation method in the embodiments of the present disclosure will be described in detail below.

In one possible embodiment, at least one pre-polymerization dimension combination may be determined before receiving the data acquisition request. In determining the pre-polymerization dimensional combination, any of the following methods may be used:

the method A,

Arranging and combining the plurality of screening dimensions to generate a plurality of dimension combinations to be screened; and determining a pre-polymerization dimension combination from the dimension combinations to be screened based on the calling frequency and/or the historical calling times of the dimension combinations to be screened.

Illustratively, taking the screening dimension as A, B, C, D as an example, possible combinations of the dimensions to be screened, such as a, B, C, D, AB, AC, AD, BC, BD, CD, ABC, ABD, ACD, ABCD, are generated by permutation and combination, and the dimension combinations AB, AC, AD, BC, BD, CD meeting the screening condition are determined as the pre-polymerization dimension combinations from the dimension combinations to be screened by setting the calling frequency and/or the historical calling times, such as the calling frequency exceeds 5 times per month and/or the historical calling times exceeds 20 times.

That is to say, the pre-polymerization dimension combination may include all combinations formed by combining a plurality of screening dimensions, or may be a part thereof, and when other combinations are required to be used subsequently, the combinations may be combined again according to the combination in the pre-polymerization dimension combination, for example, the combinations are calculated by addition and subtraction, and the like, for example, the pre-polymerization combination includes AD and ADFG, FG may be obtained by the two combinations, which is not described in detail herein.

Therefore, the determining process of the pre-polymerization dimension combination can be automatically completed by a program, a certain number of pre-polymerization dimension combinations can be determined without manual setting, data can be subjected to polymerization processing according to the pre-polymerization dimension combinations to obtain pre-polymerization data, and the pre-polymerization data is used for serving as target data to be sent to a user sending a data acquisition request at present after the pre-polymerization dimension matching is successful.

Method B,

The selection instruction may include a first selection instruction and a second selection instruction, and based on the first selection instruction in the selection instruction, a first target dimension may be determined from a plurality of screening dimensions corresponding to the data, where the first target dimension may be a selectable dimension displayed to the data requester, and thus, the data requester may initiate a data acquisition request based on the displayed first target dimension; then, based on a second selection instruction in the selection instructions, the second target dimension may be determined from the first target dimension, and then at least one pre-polymerization dimension combination may be determined based on the first target dimension and the second target dimension, so that the number of pre-polymerization dimension combinations may be reduced, and the number of finally determined pre-polymerization dimension combinations may be maintained at a reasonable level, and a corresponding pre-polymerization processing time may not be too long due to an excessive number.

The first selection instruction and the second selection instruction can be manually set before data request is carried out by a data requester; or may be manually set by a data manager, i.e., an administrator. These two cases will be described in detail separately below.

Case one, the selection instruction is set by the data requestor.

In specific implementation, before a user (i.e., a data requester) acquires data, the user may first perform preliminary screening on a plurality of screening dimensions corresponding to the data through the first selection instruction according to own needs, and then select the most likely screening dimension according to own needs on the basis of the preliminary screening, and perform screening on the first target dimension again to obtain the second target dimension.

Illustratively, taking the data to be filtered as the page click rate as an example, the corresponding filtering dimensions include dates, school departments, grades, genders and the like. If the identity of the user is a teacher, the date, the department of study, the grade and the gender in the identity can be selected as the first target dimension which is possibly used, and further, the date, the department of study and the grade can be selected as the second target dimension which is commonly used according to the historical data acquisition condition.

And in the second case, the selection instruction is set by an administrator.

The administrator may set the same selection instruction for users with different identities, and after the user logs in the user side capable of sending the data acquisition request, the data acquisition pages carrying the first target dimension and the second target dimension displayed by each user side may be the same.

In one possible embodiment, the administrator may set different selection instructions for users of different identities. Therefore, after receiving a login request of a user side, the server can verify the identity of the user, determine the identity information of the user, match the corresponding first target dimension and second target dimension based on the identity information of the user, and then display the first target dimension and second target dimension on the user side for the user to select.

In a possible embodiment, when determining at least one pre-polymerization dimension combination based on the first target dimension and the second target dimension, as shown in fig. 2, it is possible to perform the following two steps:

s1011, determining a third target dimension of the plurality of first target dimensions except the at least one second target dimension based on the at least one second target dimension.

In practical applications, the first target dimension may include the second target dimension and a third target dimension. The screening dimension with higher user selection probability in the first target dimension may be used as the second target dimension, and the screening dimension with lower user selection probability may be used as the third target dimension.

Illustratively, A, B, C, D is taken as the first target dimension that may be used, wherein A, B, C is taken as the second target dimension that is commonly used as an example. The corresponding third target dimension is D, which can characterize the screening dimension that may be used but is not commonly used.

S1012, determining the at least one pre-polymerization dimension combination based on the third target dimension.

In a specific implementation, all the filtering dimensions included in the third target dimension may be arranged and combined, and the at least one pre-polymerization dimension combination may be generated by combining the second target dimension.

Illustratively, still taking A, B, C as the second target dimension, and taking the corresponding third target dimension as D as an example, D is arranged and combined to generate two possible combinations of selected D and unselected D, and the two pre-polymerization dimension combinations of ABCD and ABC can be obtained by combining the second target dimension.

In this way, the pre-polymerization dimension combination can be generated in a targeted manner, so that the dimension combination generated by permutation and combination can be determined as the target dimension combination without screening, and compared with the method in which the dimension combination is generated randomly and then screened based on objective historical data and the like, the method B adds a subjective selection instruction, so that the generated pre-polymerization dimension better meets the subjective requirements of a data acquirer.

Further, at least one pre-polymerization dimension combination can be determined through the steps, and after the at least one pre-polymerization dimension combination is determined, data pre-polymerization treatment can be performed based on the at least one pre-polymerization dimension combination to obtain pre-polymerization data corresponding to the at least one pre-polymerization dimension combination.

In a possible embodiment, after determining at least one pre-polymerization dimension combination, the data pre-polymerization processing is performed based on the at least one pre-polymerization dimension combination, which may be to first determine a possible value corresponding to each dimension in each pre-polymerization dimension combination, and then perform data pre-polymerization processing based on the possible value corresponding to each dimension in each pre-polymerization dimension combination.

For example, if the obtained pre-polymerization dimension combination is AB, for the dimension a, the possible value is a₁、A₂For dimension B, a possible value is B₁、B₂Then for the pre-polymerization dimension combination, there are four possible values, respectively A₁B₁、A₁B₂、A₂B₁、A₂B₂And then, data pre-polymerization treatment can be carried out according to each value under the pre-polymerization dimension combination, so that four groups of pre-polymerization and data can be obtained.

For example, the stored pre-polymerization dimension combinations and corresponding pre-polymerization data storage relationships can be as shown in table 1 below.

TABLE 1

Pre-polymerization dimensional combinations	Dimension A	Dimension B	Prepolymerization data
				A₁B₁	High School	Grade one	100
A₁B₂	High School	Second grade	60
				A₂B₁	Middle school	Grade one	80
A₂B₂	Middle school	Second grade	80

In the above table, dimension a is the academic department; dimension B is grade; pre-polymerization data as Page click, A₁The academic department is "high and middle", A₂Indicates that the department of academic institution is "junior middle school" and B₁The grade is represented as 'one grade', B₂The expression grade is 'second grade'; a. the₁B₁Representing the high, middle and one grade; a. the₁B₂Representing the second grade of high school; a. the₂B₁Representing the first-middle-school grade; a. the₂B₂Representing the junior middle school second grade. Table 1 shows an example of storing only one pre-polymerization combination AB, and the storage relationship between other pre-polymerization combinations and the corresponding pre-polymerization data is similar to that in table 1, which is not listed here.

In a specific implementation, after at least one pre-polymerization dimension combination and pre-polymerization data corresponding to the at least one pre-polymerization dimension combination are determined, at least one screening dimension corresponding to the data acquisition request may be acquired according to S101, which is described in detail as follows.

In one possible implementation, the data acquisition request may be a data acquisition request related to the first target dimension/second target dimension.

For example, after determining a first target dimension of a plurality of filtering dimensions corresponding to the data, as shown in fig. 3, a user (here, a data requestor) may be presented with a data filtering interface as shown in fig. 4, on which selectable filtering dimensions are related to the first target dimension.

Illustratively, taking the A, B, C, D as the first target dimension that may be used, wherein the A, B, C is the second target dimension that is commonly used as an example, four selectable screening dimensions of A, B, C, D may be displayed on the data screening interface, which are completely consistent with the first target dimension.

Illustratively, a user can select a required screening dimension at a user side, and set values corresponding to the selected screening dimension, where the values corresponding to different screening dimensions form a screening condition for data.

For example, taking the above-mentioned screening dimension as AB as an example, the user may select 2 screening dimensions of department of academic department and grade, and correspondingly set the value corresponding to the department of academic department as "high school", and the value corresponding to the grade as "grade one". After the setting is completed, the server can directly receive the data acquisition request carrying the screening conditions.

In a possible implementation, the selectable filtering dimension presented to the user may also be a filtering dimension of the data itself, and the dimension presented is not limited by the first target dimension and the second target dimension.

Illustratively, taking the A, B, C, D as the first target dimension that may be used, wherein the A, B, C is the second target dimension that is commonly used as an example, six selectable screening dimensions of A, B, C, D, E, F may be displayed on the data screening interface, including two screening dimensions of E and F that are not provided by the first target dimension.

In a specific implementation, after the at least one screening dimension corresponding to the data acquisition request is acquired according to S101, the at least one screening dimension corresponding to the data acquisition request and the at least one pre-polymerization dimension combination may be matched according to S102, and pre-polymerization data corresponding to a target pre-polymerization dimension combination that is successfully matched is used as target data corresponding to the data acquisition request, which is described in detail as follows.

In practical application, each pre-polymerization dimension combination often has a plurality of values, when matching is performed, at least one screening dimension corresponding to the data acquisition request and the pre-polymerization dimension combination can be matched, a target pre-polymerization dimension combination corresponding to the at least one screening dimension is determined, then the value corresponding to the at least one screening dimension is respectively matched with each value under the target pre-polymerization dimension combination, and when the values corresponding to the completely consistent target pre-polymerization dimension combination are matched, the matching is considered to be successful.

In the above example, after receiving a data request with a department of higher school and a grade of one grade, matching the data request with the pre-aggregation dimension combination in table 1, and finding out 100 fully consistent pre-aggregation data, that is, sending 100 to the user side as target data corresponding to the data acquisition request, where the user side may also display the target data in a preset display location area after receiving the target data.

In a possible implementation manner, after at least one screening dimension corresponding to the data obtaining request is matched with the at least one pre-aggregation dimension combination, if matching fails, a data attribute corresponding to the data obtaining request may be determined, where the data attribute is used to indicate whether data requested to be obtained by the data obtaining request can be aggregated, and then target data corresponding to the data obtaining request is determined according to the data attribute corresponding to the data obtaining request.

Illustratively, the aggregated data may be, for example, page browsing volume, page clicking volume, number of downloads, etc., and the non-aggregated data may be number of independent visitors, number of visits, etc.

The data attribute may be an attribute of data requested to be acquired by the data acquisition request, and two scenes, which may be aggregated and not aggregated, may be obtained according to whether the data attribute is aggregated, where corresponding operation steps in different scenes are different, and each scene will be described below.

Scene one, data attribute are polymerizable

In this scenario, based on the at least one pre-polymerization dimension combination and the at least one screening dimension, aggregation processing may be performed on pre-polymerization data corresponding to the at least one pre-polymerization dimension combination to obtain target data corresponding to the data acquisition request.

In a specific operation, when performing aggregation processing on the pre-polymerization data corresponding to the at least one pre-polymerization dimension combination to obtain the target data corresponding to the data acquisition request, as shown in fig. 5, the following two steps may be performed:

s501, determining an aggregation mode corresponding to the data acquisition request.

Wherein the polymerization mode comprises at least one of the following modes:

In a possible implementation manner, when each type of data is stored, whether the data can be aggregated or not can be manually set, and the corresponding aggregation manner, for example, the page click amount can be aggregated by addition, and when an administrator enters the data or inputs the selection instruction, the administrator can set the data or input the selection instruction, and configure the aggregation manner of the page click amount as the addition aggregation.

Here, the aggregation manner corresponding to the data acquisition request may be an aggregation manner of data corresponding to the data acquisition request, for example, if the data requested to be acquired by the data acquisition request is the number of clicks, the aggregation manner corresponding to the data acquisition request is summation.

S502, according to the corresponding aggregation mode, performing aggregation processing on the pre-aggregation data corresponding to the at least one pre-aggregation dimension combination to obtain target data corresponding to the data acquisition request.

For example, still taking the pre-polymerization combination corresponding to table 1 as an example, when the screening dimension corresponding to the received data acquisition request is "school", and the corresponding screening condition is "high school", the screening dimensions are matched; and when the matching fails, judging the page click rate, and after determining that the page click rate can be polymerized, polymerizing the page click rates in the table 1 according to a summation and polymerization mode corresponding to the page click rate to obtain the target data of 60 plus 100 plus 160.

Scene two, data attribute is non-aggregatable

In this scenario, data screening may be performed based on at least one screening dimension corresponding to the data acquisition request, so as to obtain target data corresponding to the data request.

In a specific implementation, because a completely consistent pre-polymerization dimension combination cannot be matched to directly obtain the target data, and the target data cannot be indirectly obtained through aggregation of pre-polymerization data corresponding to the pre-polymerization dimension combination, the target data can be directly screened only based on the screening dimension corresponding to the data acquisition request to obtain the target data corresponding to the data acquisition request.

In this process, since the database storing the pre-polymerization data corresponding to the pre-polymerization dimension combination may be a different database from the database storing all data, for example, the database a stores the pre-polymerization data corresponding to the pre-polymerization dimension combination, and the database a may use the engine 1 when acquiring data; the database B stores all data, and because the database B stores data with a large magnitude, when data is acquired, an engine with a good acquisition effect in the scene may be used for data acquisition, for example, the engine 2 with a better effect than the engine 1 may be used instead to search for data, so as to improve the efficiency of acquiring data in the database B, thereby implementing a data acquisition request and a corresponding data acquisition process, and drilling down from the database a on the upper layer to the database B on the lower layer.

Further, when the time consumed for data acquisition by the engine is still long, the user can be prompted to generate an acquisition task corresponding to the data acquisition request and to check the task later through modes of sending a popup window and the like.

In a possible implementation manner, after the data is screened based on the at least one screening dimension corresponding to the data acquisition request to obtain the target data corresponding to the data request, the at least one screening dimension corresponding to the data acquisition request may be further used as a pre-aggregation dimension combination, and the obtained target data may be used as pre-aggregation data corresponding to the pre-aggregation dimension combination and stored.

Specifically, since the completely consistent pre-polymerization dimension combination cannot be matched to directly obtain the target data, and the target data cannot be indirectly obtained through aggregation of pre-polymerization data corresponding to the pre-polymerization dimension combination, the method for obtaining the target data from the data to be filtered based on the filtering dimension corresponding to the data obtaining request would result in a long time consumed by the data obtaining request. Therefore, at least one screening dimension corresponding to the data acquisition request can be used as a pre-polymerization dimension combination, the obtained target data is used as pre-polymerization data corresponding to the pre-polymerization dimension combination, and the pre-polymerization data is stored, so that the target data can be directly matched when being acquired next time, and the data to be screened does not need to be screened in real time.

Through the steps, the pre-polymerization dimension combination is supplemented based on the actual use condition, so that the problem caused by careless omission in configuring the pre-polymerization dimension combination can be avoided, and the effects of saving data acquisition time and improving data acquisition efficiency are achieved.

In a possible embodiment, after determining at least one pre-polymerization dimension combination based on the first target dimension and the second target dimension, referring to fig. 6, the pre-polymerization dimension combination may be deleted by the following two steps:

s601, determining a calling frequency corresponding to the at least one pre-polymerization dimension combination.

Wherein the calling frequency may be the number of times that the call is matched to and/or used for aggregation within a preset time period.

For example, each month may be set as a standard statistical period, and the number of times that the pre-polymerization dimension combination is matched and/or used for polymerization is determined as the calling frequency corresponding to the pre-polymerization dimension combination in the month.

S602, deleting the corresponding pre-polymerization dimension combination with the calling frequency less than the preset frequency and pre-polymerization data corresponding to the pre-polymerization dimension combination.

In particular implementations, a call frequency threshold may be set for the call frequency. For example, the call frequency threshold may be set five times per month, and then the pre-polymerization dimension combination with the call frequency less than five times per month and the pre-polymerization data corresponding to the pre-polymerization dimension combination may be deleted.

Through the steps, the pre-polymerization dimension combination with the calling frequency smaller than the preset frequency and the corresponding pre-polymerization data are deleted, so that the utilization efficiency of the pre-polymerization dimension combination is maximized, on one hand, the pre-polymerization dimension combination is simplified, and therefore the time consumed in the matching process is saved, on the other hand, the data storage pressure caused by the fact that the pre-polymerization data need to be stored in advance is reduced, and therefore the effects of saving data acquisition time and improving data acquisition efficiency are achieved.

According to the data acquisition method, after the data acquisition request is received, at least one screening dimension corresponding to the data acquisition request can be matched with at least one pre-polymerization dimension combination, pre-polymerization data corresponds to the pre-polymerization dimension combination, and therefore after the pre-polymerization dimension combination is successfully matched, the pre-polymerization data corresponding to the pre-polymerization dimension combination can be directly used as target data corresponding to the data acquisition request, data screening and aggregation difficulties caused by excessive data dimensions and excessive data quantity can be avoided, the time for acquiring the target data is shortened, the corresponding target data can be quickly matched for the data acquisition request after the data acquisition request is received, and the data acquisition efficiency is improved.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

Based on the same inventive concept, a data acquisition device corresponding to the data acquisition method is also provided in the embodiments of the present disclosure, and because the principle of solving the problem of the device in the embodiments of the present disclosure is similar to the data acquisition method in the embodiments of the present disclosure, the implementation of the device may refer to the implementation of the method, and repeated details are not repeated.

Referring to fig. 7, a schematic diagram of a data acquisition apparatus provided in an embodiment of the present disclosure is shown, where the apparatus includes: an acquisition module 701 and a matching module 702; wherein the content of the first and second substances,

an obtaining module 701, configured to receive a data obtaining request, and obtain at least one screening dimension corresponding to the data obtaining request;

a matching module 702, configured to match at least one screening dimension corresponding to the data obtaining request with at least one pre-aggregation dimension combination; the prepolymerization dimension combination corresponds to prepolymerization data obtained by polymerization treatment according to the prepolymerization dimension combination; and using the pre-polymerization data corresponding to the successfully matched target pre-polymerization dimension combination as the target data corresponding to the data acquisition request.

In a possible embodiment, the matching module 702 is configured to determine at least one pre-polymerization dimension combination according to the following method:

In a possible implementation, the matching module 702, when determining at least one pre-polymerization dimension combination based on the plurality of first target dimensions and the at least one second target dimension, is configured to:

In a possible implementation, the matching module 702 is further configured to:

the matching module 702, when performing aggregation processing on the pre-polymerization data corresponding to the at least one pre-polymerization dimension combination to obtain target data corresponding to the data acquisition request, is configured to:

determining an aggregation mode corresponding to the data acquisition request;

In a possible implementation manner, if the data attribute corresponding to the data obtaining request is non-aggregatable, the matching module 702 is further configured to:

In a possible implementation manner, after performing data screening based on at least one screening dimension corresponding to the data obtaining request to obtain target data corresponding to the data request, the matching module 702 is further configured to:

In a possible implementation, the matching module 702 is further configured to:

The data acquisition device provided by the disclosure can match at least one screening dimension corresponding to a data acquisition request with at least one pre-polymerization dimension combination after receiving the data acquisition request, and since the pre-polymerization dimension combination corresponds to pre-polymerization data, after the matching is successful, the pre-polymerization data corresponding to the pre-polymerization dimension combination can be directly used as target data corresponding to the data acquisition request, so that data screening and aggregation difficulties caused by excessive data dimensions and excessive data volume can be avoided, the time for acquiring the target data is reduced, the corresponding target data can be quickly matched for the data acquisition request after receiving the data acquisition request, and the efficiency of data acquisition is improved.

The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.

Based on the same technical concept, the embodiment of the disclosure also provides computer equipment. Referring to fig. 8, a schematic structural diagram of a computer device 800 provided in the embodiment of the present disclosure includes a processor 801, a memory 802, and a bus 803. The memory 802 is used for storing execution instructions and includes a memory 8021 and an external memory 8022; the memory 8021 is also referred to as an internal memory, and is used for temporarily storing operation data in the processor 801 and data exchanged with an external storage 8022 such as a hard disk, the processor 801 exchanges data with the external storage 8022 through the memory 8021, and when the computer apparatus 800 operates, the processor 801 communicates with the storage 802 through the bus 803, so that the processor 801 executes the following instructions:

In one possible implementation, the processor 801 executes instructions that determine at least one pre-polymerization dimension combination according to the following method:

In one possible embodiment, the determining at least one pre-polymerization dimension combination based on the plurality of first target dimensions and the at least one second target dimension by the processor 801 comprises:

In a possible implementation manner, the instructions executed by the processor 801 further include:

In one possible embodiment, the processor 801 executes instructions, wherein the pre-polymerization data comprises at least one type of data;

determining an aggregation mode corresponding to the data acquisition request;

In one possible implementation, the aggregation manner includes at least one of the following manners in the instructions executed by the processor 801:

In a possible implementation manner, in the instructions executed by the processor 801, if the data attribute corresponding to the data obtaining request is non-aggregatable, the method further includes:

In a possible implementation manner, the instructions executed by the processor 801 further include, after performing data screening based on at least one screening dimension corresponding to the data acquisition request to obtain target data corresponding to the data request:

The embodiments of the present disclosure also provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the data acquisition method described in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

The embodiments of the present disclosure also provide a computer program product, where the computer program product carries a program code, and instructions included in the program code may be used to execute the steps of the data acquisition method in the foregoing method embodiments, which may be referred to specifically in the foregoing method embodiments, and are not described herein again.

The computer program product may be implemented by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A method of data acquisition, comprising:

matching at least one screening dimension corresponding to the data acquisition request with at least one pre-polymerization dimension combination; the prepolymerization dimension combination corresponds to prepolymerization data obtained by polymerization treatment according to the prepolymerization dimension combination;

and using the pre-polymerization data corresponding to the successfully matched target pre-polymerization dimension combination as the target data corresponding to the data acquisition request.

2. The method according to claim 1, characterized in that at least one pre-polymerization dimensional combination is determined according to the following method:

3. The method according to claim 1, characterized in that at least one pre-polymerization dimensional combination is determined according to the following method:

4. The method of claim 2, wherein determining at least one pre-polymerization dimension combination based on the plurality of first target dimensions and the at least one second target dimension comprises:

5. The method according to any one of claims 1 to 4, further comprising:

6. The method of claim 5, wherein the pre-polymerization data comprises at least one data;

determining an aggregation mode corresponding to the data acquisition request;

7. The method of claim 6, wherein the polymerization mode comprises at least one of:

8. The method of claim 5, wherein if the data attribute corresponding to the data acquisition request is non-aggregatable, the method further comprises:

9. The method according to claim 8, wherein after performing data screening based on at least one screening dimension corresponding to the data acquisition request to obtain target data corresponding to the data request, the method further comprises:

10. The method of claim 1, further comprising:

11. A data acquisition apparatus, comprising:

12. A computer device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when a computer device is running, the machine-readable instructions when executed by the processor performing the steps of the data acquisition method of any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, performs the steps of the data acquisition method according to any one of claims 1 to 10.