CN106446170A - Data querying method and device - Google Patents
Data querying method and device Download PDFInfo
- Publication number
- CN106446170A CN106446170A CN201610852042.7A CN201610852042A CN106446170A CN 106446170 A CN106446170 A CN 106446170A CN 201610852042 A CN201610852042 A CN 201610852042A CN 106446170 A CN106446170 A CN 106446170A
- Authority
- CN
- China
- Prior art keywords
- data
- query
- offline
- statistical
- inquiry request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
Abstract
The invention discloses a data querying method and device, and belongs to the technical field of data processing. The method comprises the steps of receiving a data querying request; determining a querying mode for which the data querying request is suitable; if the data querying request is suitable for offline statistical querying, performing querying on the offline statistical querying; if the data querying request is suitable for real-time statistical querying, performing real-time querying through a data statistical tool. According to the data querying method and device, the statistical querying time is shortened to the maximum extent, and the data querying efficiency is improved.
Description
Technical field
The present invention relates to technical field of data processing, more particularly to data query method and device.
Background technology
After reaching the standard grade with each application, if it is desired to know the service condition of user, then need to user behavior data
Counted, i.e. data statisticss.Operation maintenance personnel is analyzed to data for convenience, and conventional method is to provide the page to be opened up
Show.However, often due to data volume is too big, and query time span is larger (for example：The download that checks in a year and a day), lead
Inquiry is caused once to take oversize, and due to applying the statistics backstage at center before, as data volume increases, when inquiry is clicked on,
The page is often directly stuck, affects work efficiency.
Therefore, it is necessary to propose a kind of data query method and device, it is to avoid the occurrence of above-mentioned.
Content of the invention
Present invention is primarily targeted at proposing a kind of data query method and device, it is intended to solve that search efficiency is low to ask
Topic.
For achieving the above object, a kind of data query method that the present invention is provided, methods described includes step：Receiving data
Inquiry request；Judge the inquiry mode is suitable for by the data inquiry request；If the data inquiry request is suitable for offline statistics
During inquiry, then inquired about from offline statistical data；If the data inquiry request is suitable for real-time statistics inquiry, pass through
Data statisticss instrument real-time query.
Alternatively, before the receiving data inquiry request, methods described also includes：Statistics at least one makes a reservation for respectively
Data in time period, obtain the offline statistical data；The inquiry mode for judging that the data inquiry request is suitable for,
Including：If the offline statistical data includes the object inquired about by the data inquiry request, the data inquiry request is fitted
Offline statistical query is closed, if the offline statistical data does not include the object inquired about by the data inquiry request, the number
It is suitable for real-time statistics inquiry according to inquiry request.
Alternatively, methods described also includes step：If a portion of the object inquired about by the data inquiry request
Including in the offline statistical data, being left part not included in the offline statistical data, then from the offline statistics
In data, inquiry includes the object that inquires about described in a portion in the offline statistical data, by data statisticss instrument
The object of the inquiry of remaining part of the real-time query not included in the offline statistical data.
Alternatively, described by data statisticss instrument real-time query, including：Log-on data loading interface；Data are carried out
Real-time statistics；Statistical result is stored in data base；The result in the query time section is obtained from the data base.
Alternatively, the data for being counted at least one predetermined amount of time respectively, obtain the offline statistical data, bag
Include：Measurement period is set；According to the measurement period, timing counts the data at least one predetermined amount of time respectively, obtains
The offline statistical data.
Additionally, for achieving the above object, the present invention also proposes a kind of data query arrangement, and described device includes：Receive mould
Block, for receiving data inquiry request；Judge module, for judging inquiry mode that the data inquiry request is suitable for；From
Line data inquiry module, if be suitable for offline statistical query for the data inquiry request, enters from offline statistical data
Row inquiry；Real time data enquiry module, if for judge the data inquiry request be suitable for real-time statistics inquiry when, by number
Instrument real-time query according to statistics.
Alternatively, described device also includes：Off-line data statistical module, for counting at least one predetermined amount of time respectively
Interior data, obtain the offline statistical data；Correspondingly, the judge module, if specifically for the offline statistical data
Including the object inquired about by the data inquiry request, then the data inquiry request is suitable for offline statistical query, if described from
Line statistical data does not include the object inquired about by the data inquiry request, then the suitable real-time statistics of the data inquiry request are looked into
Ask.
Alternatively, if a portion of object that the data inquiry request is inquired about is included in the offline statistical number
According in, be left part not included in the offline statistical data, then inquiring about from the offline statistical data is included described
The object that inquires about described in a portion in offline statistical data, by data statisticss instrument real-time query not included in described
The object of the inquiry of the remaining part in offline statistical data.
Alternatively, the real time data enquiry module, including：Interface start unit, for log-on data loading interface；Real
When data statisticss unit, for carrying out real-time statistics to data；First storage unit, for being stored in data base by statistical result
In；Query unit, for obtaining the result in the query time section from the data base.
Alternatively, the off-line data statistical module, including：Arranging unit, for arranging measurement period；Off-line data is united
Meter unit, for according to the measurement period, timing counts the data at least one predetermined amount of time respectively, obtain described from
Line statistical data.
Data query method and device proposed by the present invention, when receiving data inquiry request, if data inquiry request is suitable
When closing offline statistical query, then inquired about from offline statistical data, if data inquiry request is suitable for real-time statistics inquiry,
Then pass through data statisticss instrument real-time query, so as to the data statisticss time is saved, improve efficiency data query.
Description of the drawings
The schematic flow sheet of the data query method that Fig. 1 is provided for first embodiment of the invention；
The sub-process schematic diagram of the data query method that Fig. 2 is provided for first embodiment of the invention；
The schematic flow sheet of the data query method that Fig. 3 is provided for second embodiment of the invention；
The sub-process schematic diagram of the data query method that Fig. 4 is provided for second embodiment of the invention；
Fig. 5 is the schematic flow sheet of example one in second embodiment of the invention；
Fig. 6 is the schematic flow sheet of example two in second embodiment of the invention；
The module diagram of the data query arrangement that Fig. 7 is provided for third embodiment of the invention；
Fig. 8 is the module diagram of real time data enquiry module in Fig. 7；
The module diagram of the data query arrangement that Fig. 9 is provided for fourth embodiment of the invention.
The realization of the object of the invention, functional characteristics and advantage will be described further in conjunction with the embodiments referring to the drawings.
Specific embodiment
It should be appreciated that specific embodiment described herein is not intended to limit the present invention only in order to explain the present invention.
The mobile terminal of each embodiment of the present invention is realized referring now to Description of Drawings.In follow-up description, use
For representing the suffix of such as " module ", " part " or " unit " of element only for being conducive to the explanation of the present invention, itself
Not specific meaning.Therefore, " module " mixedly can be used with " part ".
Embodiment one
As shown in figure 1, present pre-ferred embodiments propose a kind of data query method, the method comprising the steps of：
Step 110, receiving data inquiry request.
Specifically, the data inquiry request that receive user is sent by the platform page.Wherein, the data of inquiry can be looked into
Ask download of certain application etc..
Further, data inquiry request includes the object that inquires about, and the object of inquiry includes query time section and looks into
The content of inquiry.For example, in the 8 o'clock sharp receiving data inquiry request on the 20th of September in 2016, it is desirable to the download in the past one week of statistics
Amount, then one week in the past (i.e. 2016 on September 12, to 2016 on September 19) are that query time section, download is inquire about interior
Hold.
Step 120, judges that data inquiry request, if appropriate for offline statistical query, if so, then enters step 130, if it is not,
Step 140 is then entered.
Specifically, if offline statistical data includes the object inquired about by data inquiry request, data inquiry request is suitable for
Offline statistical query, and step 130 is entered, otherwise it is judged to be not suitable for offline statistical query, and enters step 140.
Further, offline statistical data includes that the data within least one time period for having counted (are for example downloaded
Amount, visit capacity etc.).Offline statistical data can be discrete, for example：Counted 2016 years January, June, August
Month download.Offline statistical data can also be periodic, for example：January to the August every month in 2016 is counted
Moon download.
Further, inquiry content be with offline statistical data identical data type (for example：The content of inquiry with
Offline statistical data is download) when, if judging query time section in data inquiry request (for example：Past one month) symbol
Polling cycle in conjunction offline database is (for example：Every month), then judge that data inquiry request is suitable for offline statistical query.
Step 130, is inquired about from offline statistical data.
Specifically, direct access statistical data from offline database, and then learn the Query Result of the data.Offline number
According to the data result for storing regular calculating in storehouse, so as to only need to read the statistics for having calculated in offline database knot
Really.That is, statistics off-line data result, equivalent to the data that reading has been stored, substantially increases search efficiency, saves
Query time.
Step 140, judges that data inquiry request is inquired about if appropriate for real-time statistics, if so, then enters step 150, if it is not,
Step 160 is then entered.
Specifically, if offline statistical data does not include the object inquired about by data inquiry request, or, if judging, data are looked into
The query time section span that askes in request is larger, then judge that data inquiry request is suitable for real-time statistics inquiry, and enter step
150, otherwise it is judged to be not suitable for real-time statistics inquiry, and enters step 160.
Step 150, by data statisticss instrument real-time query.
Preferably, the data statisticss instrument can be Spark.Spark is the Open-Source Tools of a big data analysis, and it can
To carry out big data process with reference to Alluxio, Hadoop distributed document storage system.Its core technology is elasticity distribution
Formula data set (Resilien distributed datasets), there is provided the MapReduce mould abundanter than Hadoop
Type, the efficiency of Spark than hadoop MapReduce fast 100 times in internal memory, fast 10 times in disk, with quick in internal memory
In successive ignition is carried out to data set, support complexity data mining algorithm and figure computational algorithm, be suitable for real-time statistics.
Fig. 2 is refer to, step 150 further includes step：
Step 210, log-on data loading interface.
Specifically, start the data loading interface of Spark.
Data are carried out real-time statistics by step 220.
Specifically, the data of journal file according to data loading interface, are loaded, data is counted, and is written to stream
In file.
Step 230, statistical result is stored in data base.
Specifically, the data base in the present embodiment can be DataFrame, then step 230 is specially：Stream file is kept
To in DataFrame file.
Further, in order to improve the efficiency of real-time statistics, the present embodiment also uses Alluxio distributed field system
System, the DataFrame data of result is saved in Alluxio and is managed.
Step 240, obtains the result of inquiry request time period from data base.
Step 160, judges that data inquiry request is inquired about if appropriate for offline statistical query and real-time statistics, if so, then enters
Enter step 170.
Specifically, if a portion of object that data inquiry request is inquired about includes, in offline statistical data, to remain
Lower part judges that data inquiry request is suitable for offline statistical query and real-time statistics are looked into not included in offline statistical data, then
Ask, and enter step 170.
Step 170, inquiry from offline statistical data includes the right of a portion inquiry in offline statistical data
As by the object of the inquiry of remaining part of the data statisticss instrument real-time query not included in offline statistical data.
Specifically, meet the data of offline statistics, then inquired about from offline statistical data.Meet the number of real-time statistics
According to then carrying out real-time query by data statisticss instrument.
Further, inquiry content be with offline statistical data identical data type (for example：The content of inquiry with
Offline statistical data is download) when, if judging, the query time section part in data inquiry request meets offline database
In polling cycle, then meet the data of the query time section of polling cycle in offline database, from offline statistical data
Inquired about, remaining part then carries out real-time query by data statisticss instrument.
For example, it is 15 days 20 days, the polling cycle in offline database that the query time section in data inquiry request is, then
The data of wherein 15 days are inquired about by the way of offline statistical query, and the data of 5 days are by the way of real-time statistics inquiry
Inquired about, and then obtained the inquiry data of 20 days.
The data query method of the present embodiment, when receiving data inquiry request, if judge data inquiry request be suitable for from
During line statistical query, then inquired about from offline statistical data, if judge that data inquiry request is suitable for real-time statistics inquiry,
Then pass through statistical query instrument real-time query, so as to the statistical query time is saved, improve efficiency data query.
Embodiment two
Fig. 3 is refer to, is the schematic flow sheet of the data query method that another preferred embodiment of the present invention is provided.Data are looked into
Inquiry method includes step：
Step 310, counts the data at least one predetermined amount of time respectively, obtains offline statistical data.
Specifically, counting statistics being carried out using the mr mode of mongodb, allows data carry out batch calculating in mongodb,
Result of calculation is stored directly in mongodb, and data need not be taken out by whole process.
Fig. 4 is refer to, step 310 is further included：
Step 410, arranges measurement period.
Step 420, according to measurement period, timing counts the data at least one predetermined amount of time respectively, obtains offline
Statistical data.
Step 430, statistical result is stored in data base.
Example one：To count the data instance of yesterday offline.
Fig. 5 is refer to, the method for the off-line data statistics of example one is：
Step 510, it is one day to arrange measurement period.
Step 520, obtains the original table of reported data yesterday, and carries out mr calculating, obtain the statistical result of data yesterday.
Step 530, statistical result is saved in the result table of yesterday.
Example two：With the offline statistics data instance of seven days in the past.
Fig. 6 is refer to, the method for the off-line data statistics of example two is：
Step 610, it is seven days to arrange measurement period.
Step 620, obtains the original table of seven days in the past working data, and carries out mr calculating, obtains over seven day data
Statistical result.
Step 630, statistical result is saved in in the result table of seven days.
Step 320, receiving data inquiry request.
Step 330, judges that data inquiry request, if appropriate for offline statistical query, if so, then enters step 340, if it is not,
Step 350 is then entered.
Step 340, is inquired about from offline statistical data.
Step 350, judges that data inquiry request is inquired about if appropriate for real-time statistics, if so, then enters step 360, if it is not,
Step 370 is then entered.
Step 360, by data statisticss instrument real-time query.
Step 370, judges that data inquiry request is inquired about if appropriate for offline statistical query and real-time statistics, if so, then enters
Enter step 380.
Step 380, inquiry from offline statistical data includes the right of a portion inquiry in offline statistical data
As by the object of the inquiry of remaining part of the data statisticss instrument real-time query not included in offline statistical data.
The content of above-mentioned steps 320-380 is identical with the content of step 110-170 in first embodiment, for identical
Content, the present embodiment will not be described here.
The data query method of the present embodiment, by arranging measurement period, according to measurement period, timing calculates statistical number
According to, and statistical result is stored in data base.The process of offline statistics is refined, number can have been calculated in advance by timed task
According to improve the efficiency of data query in the future.
Embodiment three
The present invention further provides a kind of data query arrangement.
The module diagram of the data query arrangement for providing for third embodiment of the invention with reference to Fig. 7, Fig. 7.
A kind of data query arrangement of the present embodiment, device includes：
Receiver module 710, for receiving data inquiry request.
Specifically, the data inquiry request that 710 receive user of receiver module is sent by the platform page.Wherein, inquiry
Data can inquire about download of certain application etc..
Further, data inquiry request includes the object that inquires about, and the object of inquiry includes query time section and looks into
The content of inquiry.For example, in the 8 o'clock sharp receiving data inquiry request on the 20th of September in 2016, it is desirable to the download in the past one week of statistics
Amount, then one week in the past (i.e. 2016 on September 12, to 2016 on September 19) are that query time section, download is inquire about interior
Hold.
Judge module 720, for judging that data inquiry request is suitable for offline statistical query or suitable real-time statistics are looked into
Ask or be suitable for offline statistical query and real-time statistics inquiry.
Specifically, if judge module 720 judges that offline statistical data includes the object inquired about by data inquiry request, sentence
Determine data inquiry request and be suitable for offline statistical query, be otherwise judged to be not suitable for offline statistical query.
Further, offline statistical data includes that the data within least one time period for having counted (are for example downloaded
Amount, visit capacity etc.).Offline statistical data can be discrete, for example：Counted 2016 years January, June, August
Month download.Offline statistical data can also be periodic, for example：January to the August every month in 2016 is counted
Moon download.
Further, inquiry content be with offline statistical data identical data type (for example：The content of inquiry with
Offline statistical data is download) when, if judging query time section in data inquiry request (for example：Past one month) symbol
Polling cycle in conjunction offline database is (for example：Every month), then judge that data inquiry request is suitable for offline statistical query.
If judge module 720 judges that if offline statistical data does not include the object inquired about by data inquiry request, or, number
Larger according to query time section span in inquiry request, then judge that data inquiry request is suitable for real-time statistics inquiry.
If judge module 720 judges that a portion of the object inquired about by data inquiry request is included in offline statistical number
According in, be left part not included in offline statistical data, then judge that data inquiry request is suitable for offline statistical query and in real time
Statistical query.
For judge module 720, off-line data enquiry module 730, if judge that data inquiry request is suitable for offline statistical query
When, then inquired about from offline statistical data.
Specifically, the direct access statistical data from offline database of off-line data enquiry module 730, and then learn the number
According to Query Result.The data result of regular calculating is stored in offline database, so as to only need to read in offline database
The statistical result for having calculated.That is, statistics off-line data result, equivalent to the data that reading has been stored, is carried significantly
High search efficiency, saves query time.
For judge module 720, real time data enquiry module 740, if judge that data inquiry request is suitable for real-time statistics inquiry
When, then pass through data statisticss instrument real-time query.
Further, if a portion of object that data inquiry request is inquired about is included in offline statistical data,
Remaining part is not included in offline statistical data, then off-line data enquiry module 730 is inquired about from offline statistical data and included
The object of a portion inquiry in offline statistical data, real time data enquiry module 740 passes through data statisticss instrument reality
When remaining part of the inquiry not included in offline statistical data inquiry object.
Further, inquiry content be with offline statistical data identical data type (for example：The content of inquiry with
Offline statistical data is download) when, if judge module 720 judges that the query time section part in data inquiry request meets
Polling cycle in offline database, then triggering off-line data enquiry module 730 is from the polling cycle for meeting offline database
Query time section data, triggering real time data enquiry module 740 inquired about from offline statistical data, and remaining part is then
Real-time query is carried out by data statisticss instrument.
Preferably, the data statisticss instrument can be Spark.Spark is the Open-Source Tools of a big data analysis, and it can
To carry out big data process with reference to Alluxio, Hadoop distributed document storage system.Its core technology is elasticity distribution
Formula data set (Resilien distributed datasets), there is provided the MapReduce mould abundanter than Hadoop
Type, the efficiency of Spark than hadoop MapReduce fast 100 times in internal memory, fast 10 times in disk, with quick in internal memory
In successive ignition is carried out to data set, support complexity data mining algorithm and figure computational algorithm, be suitable for real-time statistics.
Fig. 8 is refer to, real time data enquiry module 740 is further included：
Interface start unit 810, for log-on data loading interface.
Specifically, interface start unit 810 starts the data loading interface of Spark.
Real time data statistic unit 820, for carrying out real-time statistics to data.
Specifically, according to data loading interface, real time data statistic unit 820 loads the data of journal file, to data
Counted, and be written in stream file.
First storage unit 830, for statistical result is stored in data base.
Specifically, the data base in the present embodiment can be DataFrame, then the first storage unit 830 is specifically for inciting somebody to action
Stream file is remained in DataFrame file.
Further, in order to improve the efficiency of real-time statistics, the present embodiment also uses Alluxio distributed field system
System, the DataFrame data of result are saved in Alluxio and are managed by the first storage unit 830.
Query unit 840, for obtaining the result of inquiry request time period from data base.
Further, if judge module 720 judges that data inquiry request is suitable for offline statistical query and real-time statistics inquiry
When, then triggering off-line data enquiry module 730 inquires about the data for being suitable for offline statistics from offline statistical data, and triggering is real
When data inquiry module 740 by data statisticss instrument real-time query be suitable for real-time statistics data.That is, meeting offline
The data of statistics, then trigger off-line data enquiry module 730 and inquired about from offline statistical data.Meet the number of real-time statistics
According to then triggering real time data enquiry module 740 carries out real-time query by data statisticss instrument.
For example, the query time section in data inquiry request is 20 days, and the measurement period in offline database is 15 days, then
The data of wherein 15 days are inquired about by the way of offline statistical query, and the data of 5 days are by the way of real-time statistics inquiry
Inquired about, and then obtained the inquiry data of 20 days.
The data query arrangement of the present embodiment, when 710 receiving data inquiry request of receiver module, if judge module 720
When judging that data inquiry request is suitable for offline statistical query, then triggering off-line data enquiry module 730 is from offline statistical data
Inquired about, if judge module 720 judges that data inquiry request is suitable for real-time statistics inquiry, triggering real time data inquiry mould
Block 740 passes through data statisticss instrument real-time query, so as to save the statistical query time, improves efficiency data query.
Example IV
Fourth embodiment of the invention provides a kind of data query arrangement, in the fourth embodiment, data query arrangement be
On the basis of 3rd embodiment, done improvement further, differs only in, and device also includes：
Off-line data statistical module, for counting the data at least one predetermined amount of time respectively, is counted offline
Data.
Specifically, off-line data statistical module carries out counting statistics using the mr mode of mongodb, allows data to exist
Batch calculating is carried out in mongodb, and result of calculation is stored directly in mongodb, and data need not be taken out by whole process.
Fig. 9 is refer to, off-line data statistical module is further included：
Arranging unit 910, for arranging measurement period.
Off-line data statistic unit 920, for according to measurement period, timing is counted at least one predetermined amount of time respectively
Data, obtain offline statistical data.
Second storage unit 930, for statistical result is stored in data base.
The data query arrangement of the present embodiment, arranges measurement period by arranging unit 910, according to measurement period, offline
920 timing of data statisticss unit calculates statistical data, and the second storage unit 930 is stored in statistical result in data base, refinement
The process of offline statistics, can calculate data in advance by timed task, improve the efficiency of data query in the future.
It should be noted that herein, term " including ", "comprising" or its any other variant are intended to non-row
The including of his property, so that a series of process including key elements, method, article or device not only include those key elements, and
And also include other key elements for being not expressly set out, or also include intrinsic for this process, method, article or device institute
Key element.In the absence of more restrictions, the key element for being limited by sentence "including a ...", it is not excluded that including to be somebody's turn to do
Also there is other identical element in the process, method of key element, article or device.
The embodiments of the present invention are for illustration only, do not represent the quality of embodiment.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can add the mode of required general hardware platform by software to realize, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.Based on such understanding, technical scheme is substantially done to prior art in other words
The part for going out contribution can be embodied in the form of software product, and the computer software product is stored in a storage medium
In (as ROM/RAM, magnetic disc, CD), use so that a station terminal equipment including some instructions (can be mobile phone, computer, clothes
Business device, air-conditioner, or network equipment etc.) execute method described in each embodiment of the present invention.
The preferred embodiments of the present invention are these are only, the scope of the claims of the present invention is not thereby limited, every using this
Equivalent structure or equivalent flow conversion that bright description and accompanying drawing content are made, or directly or indirectly it is used in other related skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of data query method, it is characterised in that methods described includes step：
Receiving data inquiry request；
Judge the inquiry mode is suitable for by the data inquiry request；
If the data inquiry request is suitable for offline statistical query, inquired about from offline statistical data；
If the data inquiry request is suitable for real-time statistics inquiry, pass through data statisticss instrument real-time query.
2. data query method according to claim 1, it is characterised in that
Before the receiving data inquiry request, methods described also includes：
Count the data at least one predetermined amount of time respectively, obtain the offline statistical data；
The inquiry mode for judging that the data inquiry request is suitable for, including：
If the offline statistical data includes the object inquired about by the data inquiry request, the data inquiry request is suitable for
Offline statistical query, if the offline statistical data does not include the object inquired about by the data inquiry request, the data
Inquiry request is suitable for real-time statistics inquiry.
3. data query method according to claim 2, it is characterised in that methods described also includes step：
If a portion of the object inquired about by the data inquiry request is included in the offline statistical data, remaining portion
Divide not included in the offline statistical data, then inquiring about from the offline statistical data is included in the offline statistical data
In a portion described in inquire about object, by data statisticss instrument real-time query not included in the offline statistical data
In remaining part the inquiry object.
4. the data query method according to claim 1 or 2 or 3, it is characterised in that described by data statisticss instrument reality
When inquire about, including：
Log-on data loading interface；
Real-time statistics are carried out to data；
Statistical result is stored in data base；
The result in the query time section is obtained from the data base.
5. data query method according to claim 2, it is characterised in that described counted at least one scheduled time respectively
Data in section, obtain the offline statistical data, including：
Measurement period is set；
According to the measurement period, timing counts the data at least one predetermined amount of time respectively, obtains the offline statistics
Data.
6. a kind of data query arrangement, it is characterised in that described device includes：
Receiver module, for receiving data inquiry request；
Judge module, for judging inquiry mode that the data inquiry request is suitable for；
Off-line data enquiry module, if be suitable for offline statistical query for the data inquiry request, from offline statistical number
According in inquired about；
Real time data enquiry module, if be suitable for real-time statistics inquiry for the data inquiry request, passes through data statisticss
Instrument real-time query.
7. data query arrangement according to claim 6, it is characterised in that described device also includes：
Off-line data statistical module, for counting the data at least one predetermined amount of time respectively, obtains the offline statistics
Data；
Correspondingly, specifically for the offline statistical data, the judge module, if include that the data inquiry request is inquired about
Object, then the data inquiry request be suitable for offline statistical query, if the offline statistical data does not include that the data are looked into
The inquired about object of request is ask, then the data inquiry request is suitable for real-time statistics inquiry.
8. data query arrangement according to claim 7, it is characterised in that if the data inquiry request inquired about right
A portion of elephant includes, in the offline statistical data, to be left part not included in the offline statistical data, then
From the offline statistical data, inquiry includes the object that inquires about described in a portion in the offline statistical data, leads to
Cross the object of the inquiry of remaining part of the data statisticss instrument real-time query not included in the offline statistical data.
9. the data query arrangement according to claim 6 or 7 or 8, it is characterised in that the real time data enquiry module,
Including：
Interface start unit, for log-on data loading interface；
Real time data statistic unit, for carrying out real-time statistics to data；
First storage unit, for statistical result is stored in data base；
Query unit, for obtaining the result in the query time section from the data base.
10. data query arrangement according to claim 7, it is characterised in that the off-line data statistical module, including：
Arranging unit, for arranging measurement period；
Off-line data statistic unit, for according to the measurement period, timing is counted at least one predetermined amount of time respectively
Data, obtain the offline statistical data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610852042.7A CN106446170A (en) | 2016-09-27 | 2016-09-27 | Data querying method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610852042.7A CN106446170A (en) | 2016-09-27 | 2016-09-27 | Data querying method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106446170A true CN106446170A (en) | 2017-02-22 |
Family
ID=58170257
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610852042.7A Pending CN106446170A (en) | 2016-09-27 | 2016-09-27 | Data querying method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106446170A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108446369A (en) * | 2018-03-15 | 2018-08-24 | 深圳市中科新业信息科技发展有限公司 | Real time data report form statistics method under mass data |
| CN109684352A (en) * | 2018-12-29 | 2019-04-26 | 江苏满运软件科技有限公司 | Data analysis system, method, storage medium and electronic equipment |
| CN110442598A (en) * | 2019-07-22 | 2019-11-12 | 阿里巴巴集团控股有限公司 | A kind of data query method and apparatus |
| CN110619006A (en) * | 2019-08-30 | 2019-12-27 | 上海曼斯克物联网科技有限公司 | Statistical data management method, device, platform and storage medium based on Internet of things |
| CN110633306A (en) * | 2019-08-14 | 2019-12-31 | 中国平安人寿保险股份有限公司 | Service data processing method and device, computer equipment and storage medium |
| CN110888917A (en) * | 2019-11-21 | 2020-03-17 | 深圳乐信软件技术有限公司 | Batch running task execution method, device, server and storage medium |
| CN111339134A (en) * | 2020-02-11 | 2020-06-26 | 广州众赢科技有限公司 | Data query method and device |
| CN111782409A (en) * | 2020-08-17 | 2020-10-16 | 支付宝(杭州)信息技术有限公司 | Task processing method, risk identification task processing device and electronic equipment |
| CN113377777A (en) * | 2021-06-29 | 2021-09-10 | 深圳市华曦达科技股份有限公司 | Data loading method, data loading device, computer program product and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436510A (en) * | 2011-12-30 | 2012-05-02 | 浙江乐得网络科技有限公司 | Method and system for improving on-line real-time search quality by off-line query |
| EP2524328A1 (en) * | 2010-01-15 | 2012-11-21 | AB Initio Technology LLC | Managing data queries |
| CN103927346A (en) * | 2014-03-28 | 2014-07-16 | 浙江大学 | Query connection method on basis of data volumes |
| CN105677836A (en) * | 2016-01-05 | 2016-06-15 | 北京汇商融通信息技术有限公司 | Big data processing and solving system simultaneously supporting offline data and real-time online data |
| CN105787132A (en) * | 2016-03-31 | 2016-07-20 | 畅捷通信息技术股份有限公司 | Method and system for controlling user behavior analysis |
-
2016
- 2016-09-27 CN CN201610852042.7A patent/CN106446170A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2524328A1 (en) * | 2010-01-15 | 2012-11-21 | AB Initio Technology LLC | Managing data queries |
| CN102436510A (en) * | 2011-12-30 | 2012-05-02 | 浙江乐得网络科技有限公司 | Method and system for improving on-line real-time search quality by off-line query |
| CN103927346A (en) * | 2014-03-28 | 2014-07-16 | 浙江大学 | Query connection method on basis of data volumes |
| CN105677836A (en) * | 2016-01-05 | 2016-06-15 | 北京汇商融通信息技术有限公司 | Big data processing and solving system simultaneously supporting offline data and real-time online data |
| CN105787132A (en) * | 2016-03-31 | 2016-07-20 | 畅捷通信息技术股份有限公司 | Method and system for controlling user behavior analysis |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108446369A (en) * | 2018-03-15 | 2018-08-24 | 深圳市中科新业信息科技发展有限公司 | Real time data report form statistics method under mass data |
| CN109684352A (en) * | 2018-12-29 | 2019-04-26 | 江苏满运软件科技有限公司 | Data analysis system, method, storage medium and electronic equipment |
| CN109684352B (en) * | 2018-12-29 | 2020-12-01 | 江苏满运软件科技有限公司 | Data analysis system, data analysis method, storage medium, and electronic device |
| CN110442598A (en) * | 2019-07-22 | 2019-11-12 | 阿里巴巴集团控股有限公司 | A kind of data query method and apparatus |
| CN110633306A (en) * | 2019-08-14 | 2019-12-31 | 中国平安人寿保险股份有限公司 | Service data processing method and device, computer equipment and storage medium |
| CN110619006A (en) * | 2019-08-30 | 2019-12-27 | 上海曼斯克物联网科技有限公司 | Statistical data management method, device, platform and storage medium based on Internet of things |
| CN110888917A (en) * | 2019-11-21 | 2020-03-17 | 深圳乐信软件技术有限公司 | Batch running task execution method, device, server and storage medium |
| CN111339134A (en) * | 2020-02-11 | 2020-06-26 | 广州众赢科技有限公司 | Data query method and device |
| CN111339134B (en) * | 2020-02-11 | 2024-03-08 | 广州拉卡拉信息技术有限公司 | Data query method and device |
| CN111782409A (en) * | 2020-08-17 | 2020-10-16 | 支付宝(杭州)信息技术有限公司 | Task processing method, risk identification task processing device and electronic equipment |
| CN111782409B (en) * | 2020-08-17 | 2023-12-19 | 支付宝(杭州)信息技术有限公司 | Task processing method, device and electronic equipment, and risk identification task processing method and device |
| CN113377777A (en) * | 2021-06-29 | 2021-09-10 | 深圳市华曦达科技股份有限公司 | Data loading method, data loading device, computer program product and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106446170A (en) | Data querying method and device | |
| EP3043447B1 (en) | Distribution transformer heavy loading and overloading mid-term and short-term pre-warning analytics model | |
| CN107451861B (en) | Method for identifying user internet access characteristics under big data | |
| CN104317839B (en) | The method and apparatus for generating report form template | |
| CN103577501A (en) | Hot topic searching system and hot topic searching method | |
| CN108289121A (en) | The method for pushing and device of marketing message | |
| CN108416375B (en) | Work order classification method and device | |
| CN106372113A (en) | News content pushing method and system | |
| CN102681999A (en) | Method and device for collecting and sending user action information | |
| CN101576988A (en) | Credit data interactive system and interactive method | |
| CN107918618B (en) | Data processing method and device | |
| CN103473036B (en) | A kind of input method skin method for pushing and system | |
| WO2021017293A1 (en) | Rule training method, apparatus, device, and storage medium | |
| CN108596785A (en) | Processing method, device, computer equipment and the storage medium of power equipment data | |
| CN108647329A (en) | Processing method, device and the computer readable storage medium of user behavior data | |
| CN110175264A (en) | Construction method, server and the computer readable storage medium of video user portrait | |
| CN107832291A (en) | Client service method, electronic installation and the storage medium of man-machine collaboration | |
| CN106709851A (en) | Big data retrieval method and apparatus | |
| CN107483381A (en) | The monitoring method and device of interlock account | |
| CN105005585A (en) | Log data processing method and device | |
| CN111382182A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN111061837A (en) | Topic identification method, device, equipment and medium | |
| CN104598780A (en) | Account identification method and system | |
| CN111507479A (en) | Feature binning method, device, equipment and computer-readable storage medium | |
| CN109829098A (en) | Search result optimization method, device and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170222 |
|
| RJ01 | Rejection of invention patent application after publication |