CN103064933B - Data query method and system - Google Patents
Data query method and system Download PDFInfo
- Publication number
- CN103064933B CN103064933B CN201210566137.4A CN201210566137A CN103064933B CN 103064933 B CN103064933 B CN 103064933B CN 201210566137 A CN201210566137 A CN 201210566137A CN 103064933 B CN103064933 B CN 103064933B
- Authority
- CN
- China
- Prior art keywords
- field
- data
- query
- acquisition unit
- data acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000012545 processing Methods 0.000 claims abstract description 20
- 230000008569 process Effects 0.000 claims description 26
- 238000012216 screening Methods 0.000 claims description 10
- 239000000284 extract Substances 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 14
- 238000013507 mapping Methods 0.000 description 11
- 230000000694 effects Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 230000001737 promoting effect Effects 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 238000012550 audit Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000004148 unit process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/93—Document management systems
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to a kind of data query method and system, described method includes: receives the inquiry request of input, carries the query word in the field to inquire about and described field in described inquiry request；From the centralized indexes table that described field is corresponding, inquiry obtains the harvester mark that described query word is corresponding；The querying command carrying described field and query word is generated according to described inquiry request, and described querying command is sent to the data acquisition unit that described harvester mark is corresponding, the data obtaining matching with the query word that carries in described querying command inquired about by local index table corresponding to the field carried by querying command described in described data acquisition unit in order to described data acquisition unit；Receive the described data that described data acquisition unit returns, form the Query Result of described inquiry request according to the data received and export.The present invention can improve the processing speed of data query, and the system resource alleviating data acquisition unit takies and the load pressure of data query server.
Description
Technical field
The present invention relates to technical field of the computer network, particularly relate to a kind of data query method, data query server, data acquisition unit and a kind of data query system.
Background technology
In the epoch that current internet is extremely flourishing, the collection of data has been widely used with inquiry system tool, various information technology (InformationTechnology, IT) system, the network equipment, safety equipment all can produce the data such as substantial amounts of daily record, a lot of daily record data is wherein had to need long term archival, and for various audits and inquiry.
In the collection inquiry system of mass data, the framework of system has distributed storage and centralised storage two ways, no matter which kind of mode, all suffers from the demand of the quick storage of massive logs data, quick search.
Existing a kind of distributed data inquiry system includes a data query server and multiple stage data acquisition unit, data acquisition unit is responsible for the collection (reception, formatting, merger) of daily record, is stored and index, and data query server is the unified entrance of log query.When needing inquiry to specify daily record, data query server issue querying command to all of data acquisition unit, the Query Result of all data acquisition units is received and collects final Query Result afterwards.If data acquisition unit is a lot, the daily record inquired about in one query exists only in minority data acquisition unit, query manipulation is very frequent again, then this existing scheme can increase the burden of all data acquisition units, power consumption and CPU (CentralProcessingUnit including data acquisition unit, CPU) resource, data acquisition unit is except inquiry simultaneously, also need to do the work of data receiver and warehouse-in, if query manipulation is very frequent, also can affect the acquisition performance of data acquisition unit, reduce the disposal ability that system is overall.
The log data of existing another kind of distributed data inquiry system adopts centralised storage, each data acquisition unit is only responsible for the collection (reception, formatting, merger) of daily record, is reported, log content does not preserve in this locality after data acquisition unit processes, but is reported to data query server to go storage.Data query server is after receiving the daily record that data acquisition unit reports, and centralized stores is in data base, and indexes, it is necessary to directly inquire about just passable time to log query in the data base of data query server.The mode of this data centralized stores, makes the query manipulation of daily record be only limitted in the data base of data query server and performs, do not interfere with data acquisition unit.But, owing to daily record data is concentrated in the data base leaving data query server in, data acquisition unit needs to report substantial amounts of daily record data, the load of data query server is made to be greatly increased on the one hand, also the bandwidth between data acquisition unit and data query server is consumed in a large number on the other hand, so also limiting the quantity of the data acquisition unit that a data query server can be with, the disposal ability of whole system can not be significantly high.
Summary of the invention
The invention provides a kind of data query method, data query server, data acquisition unit and data query system, the processing speed of data query can be improved, the system resource alleviating data acquisition unit takies and the load pressure of data query server, promotes the disposal ability of whole system.
For achieving the above object, first aspect present invention provides a kind of data query method, and described method includes:
Receive the inquiry request of input, described inquiry request is carried the query word in the field to inquire about and described field；
From the centralized indexes table that described field is corresponding, inquiry obtains the harvester mark that described query word is corresponding, stores the corresponding relation of the query word in described field and harvester mark in described centralized indexes table；
The querying command carrying described field and query word is generated according to described inquiry request, and described querying command is sent to the data acquisition unit that described harvester mark is corresponding, in the local index table that the field carried in order to described data acquisition unit querying command described in described data acquisition unit is corresponding, inquiry obtains the data matched with the query word that carries in described querying command；
Receive the described data that described data acquisition unit returns, form the Query Result of described inquiry request according to the data received and export.
In conjunction with first aspect, in the first possible embodiment of first aspect, before the described centralized indexes table corresponding from described field, inquiry obtains the harvester mark that described query word is corresponding, also include:
For described field, set up the centralized indexes table that described field is corresponding；
Described set up the centralized indexes table that described field is corresponding, including:
What receive the described field that each data acquisition unit sends reports concordance list, described in report concordance list include sending described in report the data in the data acquisition unit of concordance list corresponding to the query word of described field；
In the centralized indexes table of described field, what the mark of storage data acquisition unit and described data acquisition unit reported reports the corresponding relation of the query word of this field in concordance list.
In conjunction with first aspect, in the embodiment that the second of first aspect is possible, the described centralized indexes table corresponding from described field, inquiry obtains the harvester mark that described query word is corresponding, including:
If described inquiry request is carried the field that at least two to be inquired about, obtain the query word of each field in described inquiry request, and record the logical relation between the query word of described each field；
The harvester mark that the query word obtaining described each field is corresponding is inquired about from the centralized indexes table that described each field is corresponding；
The logical relation between query word according to described each field, from the described harvester mark that inquiry obtains, screening is met the harvester mark of described logical relation.
Second aspect, present invention also offers a kind of data query method, and described method includes:
Receiving the querying command that data query server sends, described querying command includes the query word in the inquiry request that described data query server receives in the field to inquire about carried and described field；
From the local index table that described field is corresponding, inquiry obtains the storage position of the data matched with the query word in described querying command, stores the corresponding relation of the query word in described field and the storage position of described data in described local index table；
Storage position according to described data, obtains described Data Concurrent and gives described data query server.
In conjunction with second aspect, in the first possible embodiment of second aspect, before the described local index table corresponding from described field, inquiry obtains the storage position of data matched with the query word in described querying command, also include:
For described field, set up the local index table that described field is corresponding；
Described set up the local index table that described field is corresponding, including:
Obtaining the storage position of the data in Current data acquisition device and described data, described data include the content of at least one field；
For each field, using described data in the content of this field as the query word of described data, set up the mapping relations of described query word and described storage position, form this field local index table at described Current data acquisition device.
The first possible embodiment in conjunction with second aspect, in the embodiment that the second of second aspect is possible, in the described local index table of field described in described Current data acquisition device, after storing the corresponding relation of the query word of described data and the storage position of described data, also include:
From the local index table of this field, extract described query word, described query word carried out duplicate removal process, formed described Current data acquisition device described field report concordance list；
The concordance list that reports of described field is sent to described data query server, sets up, in order to described data query server, the centralized indexes table that described field is corresponding.
In conjunction with second aspect, in the third possible embodiment of second aspect, the described local index table corresponding from described field, inquiry obtains the storage position of the data matched with the query word in described querying command, including:
If described querying command carries the field that at least two to be inquired about, obtain the query word of each field in described querying command, and record the logical relation between the query word of described each field；
From the local index table that described each field is corresponding, inquiry obtains the storage position of the data matched with the query word of each field in described querying command；
The logical relation between query word according to described each field, from the storage position inquiring about the described data obtained, screening is met the storage position of the data of described logical relation.
The third aspect, present invention also offers a kind of data query server, and described data query server includes:
First receives unit, for receiving the inquiry request of input, carries the query word in the field to inquire about and described field in described inquiry request；
First query unit, for in the centralized indexes table corresponding from described field, inquiry obtains described first and receives the harvester mark that the query word that carries of described inquiry request that unit receives is corresponding, stores the corresponding relation of the query word in described field and harvester mark in described centralized indexes table；
First processing unit, for generating the querying command carrying described field and query word according to described inquiry request, and described querying command is sent to the data acquisition unit of the harvester mark correspondence that described first query unit inquiry obtains, in the local index table that the field carried in order to described data acquisition unit querying command described in described data acquisition unit is corresponding, inquiry obtains the data matched with the query word that carries in described querying command；
First output unit, for receiving the described data that described data acquisition unit returns, forms the Query Result of described inquiry request according to the data received and exports.
In conjunction with the third aspect, in the first possible embodiment of the third aspect, described data query server also includes:
First indexing units, for for described field, setting up the centralized indexes table that described field is corresponding；
Described first indexing units includes:
First receives subelement, reports concordance list for what receive described field that each data acquisition unit sends, described in report concordance list include sending described in report the data in the data acquisition unit of concordance list corresponding to the query word of described field；
First index subelement, in the centralized indexes table of described field, what the mark of storage data acquisition unit and described data acquisition unit reported reports the corresponding relation of the query word of this field in concordance list.
In conjunction with the third aspect, in the embodiment that the second of the third aspect is possible, described first query unit includes:
First resolves subelement, carrying, if receiving in the described inquiry request that unit receives for described first, the field that at least two to be inquired about, obtaining the query word of each field in described inquiry request, and recording the logical relation between the query word of described each field；
First inquiry subelement, obtains the harvester mark that the query word of described each field of described first parsing subelement acquisition is corresponding for inquiry the centralized indexes table corresponding from described each field；
First filters subelement, for the logical relation between the query word of the described each field obtained according to described first parsing subelement, inquire about from described first and the described harvester mark that subelement inquiry obtains is screened the harvester mark being met described logical relation.
Fourth aspect, present invention also offers a kind of data acquisition unit, and described data acquisition unit includes:
Second receives unit, and for receiving the querying command that data query server sends, described querying command carries the query word in the field to inquire about and described field；
Second query unit, obtain and the described second storage position receiving the data that the query word in the querying command that unit receives matches for inquiry in the local index table corresponding from described field, described local index table stores the corresponding relation of the query word in described field and the storage position of described data；
Second processing unit, the storage position of the described data for obtaining according to described second query unit inquiry, obtain described Data Concurrent and give described data query server.
In conjunction with fourth aspect, in the first possible embodiment of fourth aspect, described data acquisition unit also includes:
Second indexing units, for for described field, setting up the local index table that described field is corresponding；
Described second indexing units includes:
Obtaining subelement, be used for the storage position of data and the described data obtaining in Current data acquisition device, described data include the content of at least one field；
Second index subelement, for each field obtained for described acquisition subelement, using described data in the content of this field as the query word of described data, in the local index table of field described in described data acquisition unit, store the corresponding relation of the query word of described data and the storage position of described data.
In conjunction with the first possible embodiment of fourth aspect, in the embodiment that the second of fourth aspect is possible, described second indexing units also includes:
3rd index subelement, extracts described query word for indexing the local index table of this field that subelement obtains from described second, described query word is carried out duplicate removal process, formed described Current data acquisition device described field report concordance list；
Sending subelement, the concordance list that reports for indexing the described field that subelement is formed by the described 3rd is sent to described data query server, sets up the centralized indexes table of described field in order to described data query server.
In conjunction with fourth aspect, in the third possible embodiment of fourth aspect, described second query unit includes:
Second resolves subelement, carrying, if receiving in the described querying command that unit receives for described second, the field that at least two to be inquired about, obtaining the query word of each field in described querying command, and recording the logical relation between the query word of described each field；
Second inquiry subelement, obtains resolving the storage position of the data that the query word of each field matches in the described querying command that subelement obtains with described second for inquiry the local index table corresponding from described each field；
Second filters subelement, for the logical relation between the query word of the described each field obtained according to described second parsing subelement, inquire about the storage position screening the data being met described logical relation the storage position of the described data that subelement inquiry obtains from described second.
5th aspect, present invention also offers a kind of data query system, and described system includes:
The data acquisition unit that the data query server of above-mentioned third aspect offer and above-mentioned fourth aspect provide.
Data query method, data query server, data acquisition unit and the data query system that the embodiment of the present invention provides, by setting up local index table and centralized indexes table in data acquisition unit and data query server respectively, the system resource that can effectively alleviate data acquisition unit takies, make data acquisition unit can have more resource for improving the performance of collection, thus promoting the disposal ability that system is overall, improve the processing speed of data query.
Accompanying drawing explanation
The Organization Chart of the data query system that Fig. 1 provides for the embodiment of the present invention；
Fig. 2 sets up the signaling diagram of process for the index that the embodiment of the present invention one provides；
A kind of data query method flow diagram that Fig. 3 provides for the embodiment of the present invention one；
Another data query method flow diagram that Fig. 4 provides for the embodiment of the present invention one；
The schematic diagram of the data query system that Fig. 5 provides for the embodiment of the present invention two；
Data query server that Fig. 6 provides for the embodiment of the present invention two and the schematic diagram of data acquisition unit；
The schematic diagram of the data query server that Fig. 7 provides for the embodiment of the present invention three；
The schematic diagram of the data acquisition unit that Fig. 8 provides for the embodiment of the present invention three.
Detailed description of the invention
Below by drawings and Examples, technical scheme is described in further detail.
Fig. 1 is the Organization Chart of the data query system that the embodiment of the present invention provides, as shown in Figure 1, the present invention adopts distributed structure/architecture, including a data query server 10 and multiple stage data acquisition unit 20, data acquisition unit 20 is responsible for the collection to data such as the massive logs that Log Source 30 reports (including reception, formatting, merger), storage and index etc. and is processed, and data query server 10 is the unified entrance of data query.
Data query method provided by the invention can be used for the quick search of mass data, illustrates for daily record data in the following embodiments.
Embodiment one
Before carrying out the inquiry of daily record data, in advance the data stored in system need to be indexed, generally carry out when data store, carry out the inquiry of data in order to system according to the concordance list set up.
The present embodiment is set up respectively local index table and centralized indexes table in data acquisition unit and data query server.Local index table is for storing the index of daily record data in Current data acquisition device, and its effect is: when providing querying condition, it is possible to what find qualified all daily records in local data specifically stores position.The index of centralized indexes table query word with harvester mark for storing each field, its effect is: when providing querying condition, data to be checked can be found and be potentially stored on which data acquisition unit, centralized indexes table gives the identification information that storage has the data acquisition unit of data to be checked.
Fig. 2 is the signaling diagram that process set up in the index that the present embodiment provides, as in figure 2 it is shown, include:
Step S101, data acquisition unit obtain the storage position of the data in Current data acquisition device and described data.
Alternatively, in data acquisition unit, the data of storage are the log data that Log Source reports.After Log Source reports log data to arrive data acquisition unit, data acquisition unit also needs to build local index for log data.
Log data is formatted by data acquisition unit and merger processes, log data is processed for the form (i.e. each row record in log sheet) of every record in log sheet, each log sheet is likely to there is multiple field, as shown in table 1 below, this log sheet includes the field such as field 1 and field 2, and sequence number represents the storage position of data.
Table 1
| Sequence number | Field 1 content | Field 2 content | ... |
| 1 | bbb | fff | ... |
| 2 | aaa | ccc | ... |
| 3 | aaa | ggg | ... |
Step S102, data acquisition unit are for each field, using described data in the content of this field as the query word of described data, set up the mapping relations of described query word and described storage position, form this field local index table at described Current data acquisition device.
In the local index of data acquisition unit, setting up local index table respectively for each field of every record in log sheet, in the log sheet that each concordance list correspondence is specified, certain field contents and the data containing described certain field contents specified are at the storage positional information of log sheet.Wherein, field contents is as the query word of corresponding data.The mapping relations of described query word and described storage position can be, but not limited to adopt the form of form to represent, the local index table for field in data acquisition unit 1 as shown in table 2 below:
Table 2
| Field 1 content | Position |
| aaa | 2 |
| aaa | 3 |
| bbb | 1 |
Data particular location on this data acquisition unit can be quickly found by the local index table of corresponding field.Such as, the particular location of certain content of field 1, then the local index table according to field 1, the position of correspondence can be quickly found out.
Step S103, data acquisition unit extract described query word from the local index table of this field, and described query word is carried out duplicate removal process, formed described Current data acquisition device described field report concordance list.
The data acquisition unit local index table according to field, extract report report concordance list.Report the concrete original log position not had each index content corresponding in concordance list, and only each data to should the content of field, i.e. query word.Generally before reporting, also query word can be carried out duplicate removal process, make the not duplicate contents of each field reported.Table 2 such as, has in field 1 a two aaa and bbb, after duplicate removal processes, reports an only one of which aaa and bbb in concordance list, as shown in table 3 below.
Table 3
| Field 1 content |
| aaa |
| bbb |
| ... |
For the query word extracted in newly-increased local index table, by its with report report concordance list phase comparison, if query word is identical, then be not added to report in concordance list, only upload and newly-increased unduplicated report concordance list.
The concordance list that reports of described field is sent to described data query server by step S104, data acquisition unit.
The as shown in table 3 concordance list that reports is reported data query server by data acquisition unit.For local index table has more news, then only upload and report content newly-increased in concordance list according to what the local index table after updating generated.
The concordance list that reports that data query server sends for multiple data acquisition units collects, and sets up the centralized indexes table that each field is corresponding in data query server respectively, specifically includes:
Step S105, data query server receive each data acquisition unit send described field report concordance list.
The data that concordance list includes in this data acquisition unit that report of data acquisition unit correspond to the query word of described field.
Step S106, set up the mapping relations of described query word and described harvester mark, form the centralized indexes table of described field.
For different fields, data query server sets up the centralized indexes table of corresponding field respectively.Such as field 1, the centralized indexes table of data query server can store the harvester mark of query word and the correspondence comprised on each data acquisition unit, as shown in table 4 below:
Table 4
| Field 1 content | Harvester ID |
| aaa | Harvester 1 |
| aaa | Harvester 2 |
| bbb | Harvester 1 |
Again such as, for field 2, the centralized indexes table that data query server is set up is as shown in table 5 below:
Table 5
| Field 2 content | Harvester ID |
| ccc | Harvester 1 |
| ddd | Harvester 2 |
| eee | Harvester 3 |
In data query server, corresponding each field has a centralized indexes table, the concordance list that reports reported by each data acquisition unit is aggregated in the centralized indexes table of data query server, namely, on data query server in the centralized indexes table of described field, what the mark of storage data acquisition unit and described data acquisition unit reported reports the corresponding relation of the query word of this field in concordance list.
After indexing table, when receiving inquiry request, can find, by concordance list, the data needing inquiry.
Fig. 3 is the data query method flow diagram that the present embodiment provides, as it is shown on figure 3, the data query method of the present invention includes:
Step S201, data query server receive the inquiry request of input.
User is retrieved by form or the mode such as expression formula retrieval, and input inquiry is asked.For form retrieval mode, and the interactive interface of user can provide fixing field, user can in the prompting frame of multiple fields input inquiry word, finally by submitting button to, inquiry request is submitted to data query server.For the mode of expression formula retrieval, then it is user's query word of directly inputting field and this field to inquire about, submits to data query server.
The inquiry request that data query server receives is carried the query word of the field to inquire about and described field.
When inquiry request comprises multiple field, the inquiry request that data query server receives also carries the logical relation between the query word of each field.Such as, user is at different search field input inquiry words, and data query server receives these logical relations between the query word and each query word of each different search field.
Such as, the inquiry request of input is: (field 1=aaa) AND (field 2=ccc), this inquiry request includes field 1 and field 2 the two field, the query word of its correspondence respectively aaa and ccc, AND represent that the logical relation between the two query word is the relation of "AND".
The inquiry from centralized indexes table corresponding to described field of step S202, data query server obtains the harvester mark that described query word is corresponding.
When inquiry request includes field 1, then the centralized indexes table of field 1 obtaining the query word matched with query word, obtaining the harvester mark of correspondence, thus obtaining the described query word data acquisition unit at which.
Such as, inquiry request is: field 1=aaa, then from the centralized indexes table shown in table 4, inquiry obtains harvester corresponding for aaa and is designated harvester 1 and harvester 3.
When inquiry request includes multiple field, this step specifically includes:
Step S2021, obtain the query word of each field in described inquiry request, and record the logical relation between the query word of the described each field carried in described inquiry request.
Step S2022, from centralized indexes table corresponding to described each field inquiry obtain the harvester mark that the query word of described each field is corresponding.
Step S2023, according to the logical relation between the query word of described each field, the described harvester mark screening obtained from step S2022 inquiry is met the harvester mark of described logical relation.
Such as, inquiry request is: (field 1=aaa) AND (field 2=ccc), then from the centralized indexes table shown in table 4, inquiry obtains harvester corresponding for aaa and is designated harvester 1 and harvester 3, from the centralized indexes table shown in table 5, inquiry obtains harvester corresponding for ccc and is designated harvester 1, logical relation between aaa and ccc is "AND", then can filter out the harvester mark only harvester 1 meeting logical relation.
Step S203, data query server generate the querying command carrying described field and query word according to described inquiry request, and described querying command is sent to the data acquisition unit that described harvester mark is corresponding.
The querying command that data query server generates can be identical with the inquiry request of input, it is also possible to only includes the query word that the purpose data acquisition unit sent includes.Such as, inquiry request is: (field 1=aaa) OR (field 2=ccc), then from the centralized indexes table shown in table 4, inquiry obtains harvester corresponding for aaa and is designated harvester 1 and harvester 3, from the centralized indexes table shown in table 5, inquiry obtains harvester corresponding for ccc and is designated harvester 1, logical relation between aaa and ccc is "or", the querying command sent to harvester 1 is: (field 1=aaa) OR (field 2=ccc), and the querying command sent to harvester 3 is: field 2=ccc.
Described data acquisition unit inquires about the data obtaining matching with described query word by the local index table of described data acquisition unit, and concrete process is follow-up to be described in detail in conjunction with Fig. 3.
Step S204, data query server receive the described data that described data acquisition unit returns, and form the Query Result of described inquiry request according to the data received and export.
The data received are collected by data query server, it is possible to but be not limited to export in table form.
Such as, for inquiry request being: (field 1=aaa) AND (field 2=ccc), the Query Result finally exported is as shown in table 6 below:
Table 6
| Sequence number | Field 1 | Field 2 | Field 3 |
| 2 | aaa | ccc | iii |
Fig. 4 is the another data query method flow diagram that the present embodiment provides, and as shown in Figure 4, the data query method of the present invention includes:
Step S301, data acquisition unit receive the querying command that data query server sends.
Described querying command includes the query word of the field to inquire about and the described field carried in the inquiry request that described data query server receives, alternatively it is also possible to the logical relation included between the query word of multiple field and described query word.
The inquiry from local index table corresponding to described field of step S302, data acquisition unit obtains the storage position of the data matched with described querying command.
When comprising the query word of multiple field in querying command, this step specifically includes:
The query word of each field in the querying command that step S3021, data acquisition unit obtaining step S301 receive, and record the logical relation between the query word of each field carried in described querying command.
The inquiry from local index table corresponding to described each field of step S3022, data acquisition unit obtains the storage position of the data matched with the query word of described each field.
Such as, querying command is: (field 1=aaa) OR (field 2=ccc), then in the local index table of field 1, inquiry obtains storage position corresponding to data is 2, and in the local index table of field 2, inquiry obtains storage position corresponding to data is also 2.
Step S3023, data acquisition unit are according to the logical relation between the query word of described each field, and from the storage position inquiring about the described data obtained, screening is met the storage position of the data of described logical relation.
The storage position of the data being met described logical relation is screened in the position that stores of the data that coupling obtains by data acquisition unit according to the logical relation between query word.
Step S303, the data acquisition unit storage position according to described data, obtains described Data Concurrent and gives described data query server.
The data acquisition unit storage position according to the step S302 data obtained, obtains corresponding described data, for instance: can acquire the data meeting described querying command is:
| Sequence number | Field 1 | Field 2 | Field 3 |
| 2 | aaa | ccc | iii |
Data acquisition unit sends the data to data query server, collects the Query Result exporting described inquiry request in order to data query server.
The data query method that the present embodiment provides, by setting up local index table and centralized indexes table in data acquisition unit and data query server respectively, when inquiring about data, the harvester mark that query word is corresponding is found by the centralized indexes table of corresponding field, thus obtaining the data of correspondence in the data acquisition unit that harvester mark is corresponding, the system resource that so can effectively alleviate data query server and data acquisition unit takies, make data acquisition unit can have more resource for improving the performance of collection, improve the processing speed of data query.
It is above the detailed description that data query method provided by the present invention is carried out, below data query system provided by the invention is described in detail.
Embodiment two
Fig. 5 is the schematic diagram of the data query system that the present embodiment provides, as it is shown in figure 5, the data query system of the embodiment of the present invention includes: data query server 10 and data acquisition unit 20.Wherein, data acquisition unit 20 is responsible for the collection of data, including receiving, formatting, merger, the data of storage are also indexed by the storage of data.Data query server 10 is used for the content of storage on multiple data acquisition units 20 is carried out unified management, and as the unified entrance of data query.
Fig. 6 is the data query server 10 that provides of the present embodiment and the schematic diagram of data acquisition unit 20, as shown in Figure 6, data query server 10 includes first indexing units the 100, first reception unit the 101, first query unit the 102, first processing unit 103 and the first output unit 104.
Data acquisition unit 20 includes the second indexing units 200, second and receives unit the 201, second query unit 202 and the second processing unit 203.
Before carrying out the inquiry of daily record data, the data stored in system need to be indexed by data acquisition unit 20 and data query server 10 in advance, generally completes when data store, and carries out the inquiry of data according to the concordance list set up in order to system.Data query server 10 utilizes the first indexing units 100 for field each described, sets up the centralized indexes table of described field.Data acquisition unit 20 utilizes the second indexing units 200 to set up the local index table of field.
Local index table is for storing the index of daily record data in Current data acquisition device, and its effect is: when providing querying condition, it is possible to what find qualified all daily records in local data specifically stores position.Centralized indexes table is for storing the index of data to be checked and harvester mark, its effect is: when providing querying condition, data to be checked can be found and be potentially stored on which data acquisition unit, centralized indexes table gives the identification information that storage has the data acquisition unit of data to be checked.
Wherein, the second indexing units 200 includes obtaining subelement the 2001, second index subelement the 2002, the 3rd index subelement 2003 and sending subelement 2004.
Obtaining the subelement 2001 storage position for the data obtaining in Current data acquisition device and described data, described data include the content of at least one field.
Alternatively, in data acquisition unit, the data of storage are the log data that Log Source reports.After Log Source reports log data to arrive data acquisition unit, data acquisition unit also needs to build local index for log data.
Log data is formatted by data acquisition unit and merger processes, log data is processed for the form (i.e. each row record in log sheet) of every record in log sheet, each log sheet is likely to there is multiple field, as shown in table 1, this log sheet includes the field such as field 1 and field 2, and sequence number represents the storage position of data.
Second index subelement 2002 is for for each field, using described data in the content of this field as the query word of described data, set up the mapping relations of described query word and described storage position, form this field local index table at described Current data acquisition device.
In the local index of data acquisition unit 20, setting up local index table respectively for each field of every record in log sheet, in the log sheet that each concordance list correspondence is specified, certain field contents and the data containing described certain field contents specified are at the storage positional information of log sheet.Wherein, field contents is as the query word of corresponding data.The mapping relations of described query word and described storage position can be, but not limited to adopt the form of form to represent, as shown in table 2.
Data particular location on this data acquisition unit can be quickly found by the local index table of corresponding field.Such as, the particular location of certain content of field 1, then the local index table according to field 1, the position of correspondence can be quickly found out.
Described query word, for extracting described query word from the local index table of this field, is carried out duplicate removal process by the 3rd index subelement 2003, formed described Current data acquisition device described field report concordance list.
The data acquisition unit local index table according to field, extract report report concordance list.Report the concrete original log position not had each index content corresponding in concordance list, and only each data to should the content of field, i.e. query word.Generally before reporting, also query word can be carried out duplicate removal process, make the not duplicate contents of each field reported.Table 2 such as, has in field 1 a two aaa and bbb, after duplicate removal processes, reports an only one of which aaa and bbb in concordance list, as shown in table 3.
For the query word extracted in newly-increased local index table, by its with report report concordance list phase comparison, if query word is identical, then be not added to report in concordance list, only upload and newly-increased unduplicated report concordance list.
Send subelement 2004 for the concordance list that reports of described field is sent to the first indexing units 100 of data query server 10, set up the centralized indexes table of described field in order to described data query server.
First indexing units 100 includes the first reception subelement 1001 and the first index subelement 1002.
First receive the described field that subelement 1001 sends for receiving each data acquisition unit report concordance list.The described data reporting concordance list to include sending in the data acquisition unit reporting concordance list are corresponding to the query word of described field.
First index subelement 1002, for setting up the mapping relations of described query word and described harvester mark, forms the centralized indexes table of described field.
First index subelement 1002, for different fields, sets up the centralized indexes table of corresponding field respectively.Such as field 1, centralized indexes table can store the harvester mark of query word and the correspondence comprised on each data acquisition unit, as shown in table 4.
The first index corresponding each field of subelement 1002 sets up a centralized indexes table, and what each data acquisition unit reported reports concordance list to be aggregated in the centralized indexes table of data query server.
After indexing table in data query server 10 and data acquisition unit 20 respectively by the first indexing units 100 and the second indexing units 200, when receiving inquiry request, can find, by concordance list, the data needing inquiry.
First receives unit 101 for receiving the inquiry request of input.
User is retrieved by form or the mode such as expression formula retrieval, and input inquiry is asked.For form retrieval mode, and the interactive interface of user can provide fixing field, user can in the prompting frame of multiple fields input inquiry word, finally by submitting button to, inquiry request is submitted to data query server.For the mode of expression formula retrieval, then it is user's query word of directly inputting field and this field to inquire about, submits to data query server.
The inquiry request that first reception unit 101 receives carries the field to inquire about and the query word of described field.
When inquiry request comprises multiple field, the inquiry request that the first reception unit 101 receives also includes the logical relation between the query word of each field.Such as, user is at different search field input inquiry words, and first receives unit 101 receives these logical relations between the query word and each query word of each different search field.
First query unit 102 is for from the centralized indexes table of described field, and inquiry obtains the harvester mark that query word that the described inquiry request that the first reception unit 101 receives carries is corresponding.
When inquiry request includes field 1, the first query unit 102 then obtains the query word matched with query word from the centralized indexes table of field 1, obtains the harvester mark of correspondence, thus obtaining the described query word data acquisition unit at which.
When inquiry request includes multiple field, the first query unit 102 includes: resolve subelement, the first inquiry subelement and first filters subelement (not shown).
First resolves subelement for obtaining the query word of each field in described inquiry request, and records the logical relation between the query word of described each field.
First inquiry subelement, obtains the harvester mark that the query word of described each field of described first parsing subelement acquisition is corresponding for inquiry the centralized indexes table corresponding from described field.
First filters subelement, and for the logical relation between the query word of the described each field obtained according to described first parsing subelement, the described harvester mark obtained from the first inquiry subelement inquiry screens the harvester mark being met described logical relation.
First processing unit 103 for generating the querying command carrying described field and query word according to described inquiry request, and described querying command is sent to the data acquisition unit 20 of the harvester mark correspondence that the first query unit 102 inquiry obtains.
The querying command that first processing unit 103 generates can be identical with the inquiry request of input, it is also possible to only includes the query word that the purpose data acquisition unit sent includes.Such as, inquiry request is: (field 1=aaa) OR (field 2=ccc), then the first query unit 102 is inquired about from the centralized indexes table shown in table 4 and is obtained harvester corresponding for aaa and be designated harvester 1 and harvester 3, first query unit 102 is inquired about from the centralized indexes table shown in table 5 and is obtained harvester corresponding for ccc and be designated harvester 1, logical relation between aaa and ccc is "or", first processing unit 103 generates the querying command sent to harvester 1: (field 1=aaa) OR (field 2=ccc), first processing unit 103 generates the querying command sent to harvester 3: field 2=ccc.
The second of data acquisition unit 20 receives unit 201 for receiving the querying command that data query server 10 sends.
Described querying command includes the query word of the field to inquire about and the described field carried in the inquiry request that data query server 10 receives, it is possible to include the logical relation between the query word of multiple field and described query word.
Second query unit 202 obtains and the second storage position receiving the data that the query word in the querying command that unit 201 receives matches for inquiry the local index table corresponding from described field.
Alternatively, during for comprising the query word of multiple field in querying command, the second query unit 202 includes: second resolve subelement, the second inquiry subelement and second filters subelement (not shown).
If the described querying command that the second parsing subelement is for the second reception unit 201 reception carries the multiple field to inquire about, obtain the query word of each field in described querying command, and record the logical relation between the query word of each field carried in described querying command.
Second inquiry subelement obtains resolving the storage position of the data that the query word of each field matches in the described querying command that subelement obtains with described second for inquiry the local index table corresponding from described each field.
Second filters subelement for the logical relation between the query word of the described each field obtained according to described second parsing subelement, inquires about the storage position screening the data being met described logical relation the storage position of the described data that subelement inquiry obtains from described second.
The storage position of the second processing unit 203 described data for obtaining according to the second query unit 202 inquiry, obtains described Data Concurrent and gives described data query server.
The data that second query unit 202 inquiry obtains are sent to the first output unit 104 of data query server 10 by the second processing unit 203, in order to export the Query Result of described inquiry request.
The described data that first output unit 104 returns for the second processing unit 203 receiving data acquisition unit 20, form the Query Result of described inquiry request according to the data received and export.
Data query server, data acquisition unit and the system that the embodiment of the present invention provides, the first indexing units is utilized to set up centralized indexes table in data query server, the second indexing units is utilized to set up local index table in data acquisition unit, it is possible to improve the processing speed of data query.When data to be checked exist only in minority data acquisition unit, the present invention need not go inquiry by every data acquisition unit, reduces the burden of data acquisition unit.For needing the log system of frequently inquiry, the scheme that the embodiment of the present invention provides can effectively alleviate the system resource of data acquisition unit and take, make data acquisition unit can have more resource for improving the performance of collection, thus promoting the disposal ability that system is overall.
Embodiment three
The schematic diagram of the data query server 10 that Fig. 7 provides for the present embodiment, as it is shown in fig. 7, described data query server 10 includes: network interface 71, processor 72 and memorizer 73.System bus 74 is used for connecting network interface 71, processor 72 and memorizer 73.
Network interface 71 is for communicating with data acquisition unit 20.
Memorizer 73 can be permanent memory, for instance hard disk drive and flash memory, has software module and device driver in memorizer 73.Software module is able to carry out the various functional modules of said method of the present invention；Device driver can be network and interface drive program.
When starting, these component softwares are loaded in memorizer 73, are then accessed by processor 72 and perform as given an order:
Receive the inquiry request of input, described inquiry request is carried the query word in the field to inquire about and described field；
From the centralized indexes table that described field is corresponding, inquiry obtains the harvester mark that described query word is corresponding, stores the corresponding relation of the query word in described field and harvester mark in described centralized indexes table；
The querying command carrying described field and query word is generated according to described inquiry request, and described querying command is sent to the data acquisition unit that described harvester mark is corresponding, the data obtaining matching with the query word that carries in described querying command inquired about by local index table corresponding to the field carried by querying command described in described data acquisition unit in order to described data acquisition unit；
Receive the described data that described data acquisition unit returns, form the Query Result of described inquiry request according to the data received and export.
The data query server of the present embodiment finds, by the centralized indexes table of field, the harvester mark that query word is corresponding, thus obtaining the data of correspondence in the data acquisition unit that harvester mark is corresponding, the system resource that can effectively alleviate data query server takies, and improves the processing speed of data query.
Further, after described processor accesses the component software of memorizer 73, perform the instruction of procedure below:
For described field, set up the centralized indexes table that described field is corresponding；
Described set up the centralized indexes table that described field is corresponding, including:
What receive the described field that each data acquisition unit sends reports concordance list, described in report concordance list include sending described in report the data in the data acquisition unit of concordance list corresponding to the query word of described field；
Set up the mapping relations of described query word and described harvester mark, form the centralized indexes table of described field.
Above-mentioned instruction process is exactly the process that data query server sets up centralized indexes table, by setting up the mapping relations of query word and harvester mark, so that when data query, find the harvester mark that query word is corresponding, and obtain the data of coupling from corresponding data acquisition unit.
Further, after described processor accesses the component software of memorizer 73, perform the instruction of procedure below:
When described inquiry request is carried the field that at least two to be inquired about, obtain the query word of each field in described inquiry request, and record the logical relation between the query word of described each field；
The harvester mark that the query word obtaining described each field is corresponding is inquired about from the centralized indexes table that described each field is corresponding；
The logical relation between query word according to described each field, the described harvester mark obtained from inquiry screens the harvester mark being met described logical relation.
Above-mentioned instruction process is exactly the process that data query server identifies for the harvester that the query word lookup of the multiple fields to inquire about is corresponding, it is possible to avoid accessing the data acquisition unit that can not fully meet inquiry request.
The schematic diagram of the data acquisition unit 20 that Fig. 8 provides for the present embodiment, as shown in Figure 8, described data acquisition unit 20 includes: network interface 81, processor 82 and memorizer 83.System bus 84 is used for connecting network interface 81, processor 82 and memorizer 83.
Network interface 81 is for communicating with data query server 10.
Memorizer 83 can be permanent memory, for instance hard disk drive and flash memory, has software module and device driver in memorizer 83.Software module is able to carry out the various functional modules of said method of the present invention；Device driver can be network and interface drive program.
When starting, these component softwares are loaded in memorizer 83, are then accessed by processor 82 and perform as given an order:
Receiving the querying command that data query server sends, described querying command includes the query word in the inquiry request that described data query server receives in the field to inquire about carried and described field；
From the local index table that described field is corresponding, inquiry obtains the storage position of the data matched with described querying command, stores the corresponding relation of the query word in described field and the storage position of described data in described local index table；
Storage position according to described data, obtains described Data Concurrent and gives described data query server.
The data acquisition unit of the present embodiment finds, by the local index table of field, the data that query word is corresponding, it is supplied to data query server, the system resource that can effectively alleviate data acquisition unit takies, make data acquisition unit can have more resource for improving the performance of collection, improve the processing speed of data query.
Further, after described processor accesses the component software of memorizer 83, perform the instruction of procedure below:
For described field, set up the local index table that described field is corresponding；
Described set up the local index table that described field is corresponding, including:
Obtaining the storage position of the data in Current data acquisition device and described data, described data include the content of at least one field；
For each field, using described data in the content of this field as the query word of described data, set up the mapping relations of described query word and described storage position, form this field local index table at described Current data acquisition device.
Above-mentioned instruction process is exactly the process that data acquisition unit sets up local index table, by setting up the mapping relations of the storage position of query word and data, thus the storage position acquisition data according to data corresponding to query word.
Further, after described processor accesses the component software of memorizer 83, perform the instruction of procedure below:
From the local index table of this field, extract described query word, described query word carried out duplicate removal process, formed described Current data acquisition device described field report concordance list；
The concordance list that reports of described field is sent to described data query server, sets up, in order to described data query server, the centralized indexes table that described field is corresponding.
Above-mentioned instruction process is exactly that data acquisition unit reports concordance list according to the foundation of local index table and is sent to the process of data query server, in order to data query server sets up centralized indexes table.
Further, after described processor accesses the component software of memorizer 83, perform the instruction of procedure below:
When described querying command carries the field that at least two to be inquired about, obtain the query word of each field in described querying command, and record the logical relation between the query word of described each field；
From the local index table that described each field is corresponding, inquiry obtains the storage position of the data matched with the query word of described each field；
The logical relation between query word according to described each field, from the storage position inquiring about the described data obtained, screening is met the storage position of the data of described logical relation.
Above-mentioned instruction process is exactly data acquisition unit for the query word of the multiple fields to inquire about finds the process of storage position of correspondence, it is possible to avoid obtaining the data that can not fully meet querying command.
Professional should further appreciate that, the unit of each example described in conjunction with the embodiments described herein and algorithm steps, can with electronic hardware, computer software or the two be implemented in combination in, in order to clearly demonstrate the interchangeability of hardware and software, generally describe composition and the step of each example in the above description according to function.These functions perform with hardware or software mode actually, depend on application-specific and the design constraint of technical scheme.Professional and technical personnel specifically can should be used for using different methods to realize described function to each, but this realization is it is not considered that beyond the scope of this invention.
The method described in conjunction with the embodiments described herein or the step of algorithm can use the software module that hardware, processor perform, or the combination of the two is implemented.Software module can be placed in any other form of storage medium known in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable ROM, depositor, hard disk, moveable magnetic disc, CD-ROM or technical field.
Above-described detailed description of the invention; the purpose of the present invention, technical scheme and beneficial effect have been further described; it is it should be understood that; the foregoing is only the specific embodiment of the present invention; the protection domain being not intended to limit the present invention; all within the spirit and principles in the present invention, any amendment of making, equivalent replacement, improvement etc., should be included within protection scope of the present invention.
Claims (11)
1. a data query method, it is characterised in that described method includes:
Data query server receives the inquiry request of user's input, carries the query word in the field to inquire about and described field in described inquiry request；
The inquiry from centralized indexes table corresponding to described field of described data query server obtains the harvester mark that described query word is corresponding, stores the corresponding relation of the query word in described field and harvester mark in described centralized indexes table；
Described data query server generates the querying command carrying described field and query word according to described inquiry request, and described querying command is sent to the data acquisition unit that described harvester mark is corresponding, in the local index table that the field carried in order to described data acquisition unit querying command described in described data acquisition unit is corresponding, inquiry obtains the data matched with the query word that carries in described querying command；
Described data query server receives the described data that described data acquisition unit returns, and forms the Query Result of described inquiry request according to the data received and exports；
Wherein, before the centralized indexes table that described data query server is corresponding from described field, inquiry obtains the harvester mark that described query word is corresponding, also include:
Described data query server, for described field, sets up the centralized indexes table that described field is corresponding；
Described set up the centralized indexes table that described field is corresponding, including:
What described data query server received the described field that each data acquisition unit sends reports concordance list, described in report concordance list include sending described in report the data in the data acquisition unit of concordance list corresponding to the query word of described field；
Described data query server is in the centralized indexes table of described field, and what the mark of storage data acquisition unit and described data acquisition unit reported reports the corresponding relation of the query word of this field in concordance list.
2. data query method according to claim 1, it is characterised in that the inquiry from centralized indexes table corresponding to described field of described data query server obtains the harvester mark that described query word is corresponding, including:
If carrying the field that at least two to be inquired about in described inquiry request, described data query server obtains the query word of each field in described inquiry request, and records the logical relation between the query word of the described each field carried in described inquiry request；
The inquiry from centralized indexes table corresponding to described each field of described data query server obtains the harvester mark that the query word of described each field is corresponding；
Described data query server is according to the logical relation between the query word of described each field, and from the described harvester mark that inquiry obtains, screening is met the harvester mark of described logical relation.
3. a data query method, it is characterised in that described method includes:
Data acquisition unit receives the querying command that data query server sends, and described querying command carries the query word in the field to inquire about and described field；
The inquiry from local index table corresponding to described field of described data acquisition unit obtains the storage position of the data matched with the query word in described querying command, stores the corresponding relation of the query word in described field and the storage position of described data in described local index table；
The described data acquisition unit storage position according to described data, obtains described Data Concurrent and gives described data query server；
Wherein, before the local index table that described data acquisition unit is corresponding from described field, inquiry obtains the storage position of data matched with the query word in described querying command, also include:
Described data acquisition unit, for described field, sets up the local index table that described field is corresponding；
Described set up the local index table that described field is corresponding, including:
Data acquisition unit obtains the storage position of the data in Current data acquisition device and described data, and described data include the content of at least one field；
Described data acquisition unit is for each field, using described data in the content of this field as the query word of described data, in the local index table of field described in described Current data acquisition device, store the corresponding relation of the query word of described data and the storage position of described data.
4. data query method according to claim 3, it is characterised in that in the described local index table of field described in described Current data acquisition device, after storing the corresponding relation of the query word of described data and the storage position of described data, also includes:
Described data acquisition unit extracts described query word from the local index table of this field, and described query word is carried out duplicate removal process, formed described Current data acquisition device described field report concordance list；
The concordance list that reports of described field is sent to described data query server by described data acquisition unit, sets up, in order to described data query server, the centralized indexes table that described field is corresponding.
5. data query method according to claim 3, it is characterised in that the inquiry from local index table corresponding to described field of described data acquisition unit obtains the storage position of the data matched with the query word in described querying command, including:
If carrying the field that at least two to be inquired about in described querying command, data acquisition unit obtains the query word of each field in described querying command, and records the logical relation between the query word of each field carried in described querying command；
The inquiry from local index table corresponding to described each field of described data acquisition unit obtains the storage position of the data matched with the query word of each field in described querying command；
Described data acquisition unit is according to the logical relation between the query word of described each field, and from the storage position inquiring about the described data obtained, screening is met the storage position of the data of described logical relation.
6. a data query server, it is characterised in that described data query server includes:
First receives unit, for receiving the inquiry request of user's input, carries the query word in the field to inquire about and described field in described inquiry request；
First query unit, for in the centralized indexes table corresponding from described field, inquiry obtains described first and receives the harvester mark that the query word that carries of described inquiry request that unit receives is corresponding, stores the corresponding relation of the query word in described field and harvester mark in described centralized indexes table；
First processing unit, for generating the querying command carrying described field and query word according to described inquiry request, and described querying command is sent to the data acquisition unit of the harvester mark correspondence that described first query unit inquiry obtains, in the local index table that the field carried in order to described data acquisition unit querying command described in described data acquisition unit is corresponding, inquiry obtains the data matched with the query word that carries in described querying command；
First output unit, for receiving the described data that described data acquisition unit returns, forms the Query Result of described inquiry request according to the data received and exports；
Described data query server also includes:
First indexing units, for for described field, setting up the centralized indexes table that described field is corresponding；
Described first indexing units includes:
First receives subelement, reports concordance list for what receive described field that each data acquisition unit sends, described in report concordance list include sending described in report the data in the data acquisition unit of concordance list corresponding to the query word of described field；
First index subelement, in the centralized indexes table of described field, what the mark of storage data acquisition unit and described data acquisition unit reported reports the corresponding relation of the query word of this field in concordance list.
7. data query server according to claim 6, it is characterised in that described first query unit includes:
First resolves subelement, if receiving in the described inquiry request that unit receives for described first and carrying the field that at least two to be inquired about, obtain the query word of each field in described inquiry request, and record the logical relation between the query word of the described each field carried in described inquiry request；
First inquiry subelement, obtains the harvester mark that the query word of described each field of described first parsing subelement acquisition is corresponding for inquiry the centralized indexes table corresponding from described each field；
First filters subelement, for the logical relation between the query word of the described each field obtained according to described first parsing subelement, inquire about from described first and the described harvester mark that subelement inquiry obtains is screened the harvester mark being met described logical relation.
8. a data acquisition unit, it is characterised in that including:
Second receives unit, and for receiving the querying command that data query server sends, described querying command carries the query word in the field to inquire about and described field；
Second query unit, obtain and the described second storage position receiving the data that the query word in the querying command that unit receives matches for inquiry in the local index table corresponding from described field, described local index table stores the corresponding relation of the query word in described field and the storage position of described data；
Second processing unit, the storage position of the described data for obtaining according to described second query unit inquiry, obtain described Data Concurrent and give described data query server；
Described data acquisition unit also includes:
Second indexing units, for for described field, setting up the local index table that described field is corresponding；
Described second indexing units includes:
Obtaining subelement, be used for the storage position of data and the described data obtaining in Current data acquisition device, described data include the content of at least one field；
Second index subelement, for each field obtained for described acquisition subelement, using described data in the content of this field as the query word of described data, in the local index table of field described in described data acquisition unit, store the corresponding relation of the query word of described data and the storage position of described data.
9. data acquisition unit according to claim 8, it is characterised in that described second indexing units also includes:
3rd index subelement, extracts described query word for indexing the local index table of this field that subelement obtains from described second, described query word is carried out duplicate removal process, formed described data acquisition unit described field report concordance list；
Sending subelement, the concordance list that reports for indexing the described field that subelement is formed by the described 3rd is sent to described data query server, sets up the centralized indexes table of described field in order to described data query server.
10. data acquisition unit according to claim 8, it is characterised in that described second query unit includes:
Second resolves subelement, if receiving in the described querying command that unit receives for described second and carrying the field that at least two to be inquired about, obtain the query word of each field in described querying command, and record the logical relation between the query word of each field carried in described querying command；
Second inquiry subelement, obtains resolving the storage position of the data that the query word of each field matches in the described querying command that subelement obtains with described second for inquiry the local index table corresponding from described each field；
Second filters subelement, for the logical relation between the query word of the described each field obtained according to described second parsing subelement, inquire about the storage position screening the data being met described logical relation the storage position of the described data that subelement inquiry obtains from described second.
11. a data query system, it is characterised in that described system includes:
Data acquisition unit as described in data query server as claimed in claims 6 or 7 and claim as arbitrary in claim 8～10.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210566137.4A CN103064933B (en) | 2012-12-24 | 2012-12-24 | Data query method and system |
| PCT/CN2013/082130 WO2014101445A1 (en) | 2012-12-24 | 2013-08-23 | Data query method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210566137.4A CN103064933B (en) | 2012-12-24 | 2012-12-24 | Data query method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103064933A CN103064933A (en) | 2013-04-24 |
| CN103064933B true CN103064933B (en) | 2016-06-29 |
Family
ID=48107563
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210566137.4A Expired - Fee Related CN103064933B (en) | 2012-12-24 | 2012-12-24 | Data query method and system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN103064933B (en) |
| WO (1) | WO2014101445A1 (en) |
Families Citing this family (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103064933B (en) * | 2012-12-24 | 2016-06-29 | 华为技术有限公司 | Data query method and system |
| CN105099735B (en) * | 2014-05-07 | 2018-05-22 | 中国移动通信集团福建有限公司 | A kind of method and system for obtaining magnanimity more detailed logging |
| CN105302827B (en) * | 2014-06-30 | 2018-11-20 | 华为技术有限公司 | A kind of searching method and equipment of event |
| CN104216957A (en) * | 2014-08-20 | 2014-12-17 | 北京奇艺世纪科技有限公司 | Query system and query method for video metadata |
| CN104317924A (en) * | 2014-10-30 | 2015-01-28 | 中国银行股份有限公司 | Data query method and device in local clearings |
| CN105871951A (en) * | 2015-01-21 | 2016-08-17 | 上海可鲁系统软件有限公司 | Industrial internet of things distributed business voucher processing method |
| CN107015990B (en) * | 2016-01-27 | 2020-06-09 | 阿里巴巴集团控股有限公司 | Data searching method and device |
| CN105930441B (en) * | 2016-04-18 | 2019-04-26 | 华信咨询设计研究院有限公司 | A kind of radio monitoring data query method |
| CN106354823A (en) * | 2016-08-30 | 2017-01-25 | 北京旷视科技有限公司 | Method, device and system for summarizing face matching system operation data |
| CN107784050A (en) * | 2016-12-14 | 2018-03-09 | 平安科技（深圳）有限公司 | Log information lookup method and device |
| CN107066610A (en) * | 2017-05-02 | 2017-08-18 | 中国联合网络通信集团有限公司 | A kind of price queries method and apparatus |
| CN107577506B (en) * | 2017-08-07 | 2021-03-19 | 台州市吉吉知识产权运营有限公司 | Data preloading method and system |
| CN109299219B (en) * | 2018-08-31 | 2022-08-12 | 北京奥星贝斯科技有限公司 | Data query method and device, electronic equipment and computer readable storage medium |
| CN109308305B (en) * | 2018-09-30 | 2021-06-08 | 广州圣亚科技有限公司 | Monitoring data query method and device and computer equipment |
| CN109299348B (en) * | 2018-11-28 | 2021-09-28 | 北京字节跳动网络技术有限公司 | Data query method and device, electronic equipment and storage medium |
| CN109885548A (en) * | 2019-02-22 | 2019-06-14 | 网易（杭州）网络有限公司 | Log inquiring method, device, storage medium and electronic device |
| CN110502915B (en) * | 2019-08-30 | 2021-07-30 | 恩亿科(北京)数据科技有限公司 | Data processing method, device and system |
| CN110674369A (en) * | 2019-09-23 | 2020-01-10 | 杭州迪普科技股份有限公司 | Data query method and device |
| CN111062193B (en) * | 2019-12-16 | 2023-04-25 | 医渡云（北京）技术有限公司 | Medical data labeling method and device, storage medium and electronic equipment |
| CN113486048A (en) * | 2021-07-13 | 2021-10-08 | 广西电力职业技术学院 | Data retrieval system and data retrieval method |
| WO2023220973A1 (en) * | 2022-05-18 | 2023-11-23 | 京东方科技集团股份有限公司 | Data processing method and apparatus, and electronic device and computer-readable storage medium |
| CN117271562B (en) * | 2023-11-21 | 2024-01-19 | 成都凌亚科技有限公司 | Data acquisition processing method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6144957A (en) * | 1997-02-28 | 2000-11-07 | Oracle Corporation | Method and apparatus for using incompatible types of indexes to process a single query |
| CN102193917A (en) * | 2010-03-01 | 2011-09-21 | 中国移动通信集团公司 | Method and device for processing and querying data |
| CN102375853A (en) * | 2010-08-24 | 2012-03-14 | 中国移动通信集团公司 | Distributed database system, method for building index therein and query method |
| CN102789487A (en) * | 2012-06-29 | 2012-11-21 | 用友软件股份有限公司 | Data query and retrieval processing device and data query and retrieval processing method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1968132B (en) * | 2006-10-16 | 2010-04-21 | 华为技术有限公司 | Method for establishing call log correlation between network entities and searching correlated call log |
| CN103064933B (en) * | 2012-12-24 | 2016-06-29 | 华为技术有限公司 | Data query method and system |
-
2012
- 2012-12-24 CN CN201210566137.4A patent/CN103064933B/en not_active Expired - Fee Related
-
2013
- 2013-08-23 WO PCT/CN2013/082130 patent/WO2014101445A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6144957A (en) * | 1997-02-28 | 2000-11-07 | Oracle Corporation | Method and apparatus for using incompatible types of indexes to process a single query |
| CN102193917A (en) * | 2010-03-01 | 2011-09-21 | 中国移动通信集团公司 | Method and device for processing and querying data |
| CN102375853A (en) * | 2010-08-24 | 2012-03-14 | 中国移动通信集团公司 | Distributed database system, method for building index therein and query method |
| CN102789487A (en) * | 2012-06-29 | 2012-11-21 | 用友软件股份有限公司 | Data query and retrieval processing device and data query and retrieval processing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103064933A (en) | 2013-04-24 |
| WO2014101445A1 (en) | 2014-07-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103064933B (en) | Data query method and system | |
| CN101853287B (en) | Data compression quick retrieval file system and method thereof | |
| CN103559217B (en) | A kind of massive multicast data towards isomeric data storehouse warehouse-in implementation method | |
| US8498995B1 (en) | Optimizing data retrieval during event data query processing | |
| CN109739867B (en) | Industrial metadata management method and system | |
| US11321315B2 (en) | Methods and systems for database optimization | |
| CN102710795B (en) | Hotspot collecting method and device | |
| CN102750326A (en) | Log management optimization method of cluster system based on downsizing strategy | |
| CN104090889A (en) | Method and system for data processing | |
| US8880463B2 (en) | Standardized framework for reporting archived legacy system data | |
| CN104573024B (en) | The self-adaptation extraction method and system of heterogeneous secure log information under a kind of complex network system | |
| US20140046928A1 (en) | Query plans with parameter markers in place of object identifiers | |
| CN101727502A (en) | Data query method, data query device and data query system | |
| CN111221791A (en) | Method for importing multi-source heterogeneous data into data lake | |
| CN111258978A (en) | Data storage method | |
| CN108509437A (en) | A kind of ElasticSearch inquiries accelerated method | |
| US11625412B2 (en) | Storing data items and identifying stored data items | |
| CN106294826A (en) | A kind of company-data Query method in real time and system | |
| WO2017000592A1 (en) | Data processing method, apparatus and system | |
| CN111858730A (en) | Data importing and exporting device, method, equipment and medium of graph database | |
| CN102955802A (en) | Method and device for acquiring data from data reports | |
| CN113364801A (en) | Management method, system, terminal device and storage medium of network firewall policy | |
| CN104239353A (en) | WEB classification control and log auditing method | |
| CN104636368A (en) | Data retrieval method and device and server | |
| KR100906454B1 (en) | Database log data management apparatus and method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20191210 Address after: 056000 Liu Han Xiang Liu Han Cun Xi, Yongnian District, Handan City, Hebei Province Patentee after: Handan Duofei Fastener Manufacturing Co.,Ltd. Address before: 510000 unit 2414-2416, building, No. five, No. 371, Tianhe District, Guangdong, China Patentee before: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd. Effective date of registration: 20191210 Address after: 510000 unit 2414-2416, building, No. five, No. 371, Tianhe District, Guangdong, China Patentee after: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd. Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd. |
|
| TR01 | Transfer of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160629 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |