CN110347722A - Data acquisition method, device, equipment and storage medium based on HBase - Google Patents
Data acquisition method, device, equipment and storage medium based on HBase Download PDFInfo
- Publication number
- CN110347722A CN110347722A CN201910625423.5A CN201910625423A CN110347722A CN 110347722 A CN110347722 A CN 110347722A CN 201910625423 A CN201910625423 A CN 201910625423A CN 110347722 A CN110347722 A CN 110347722A
- Authority
- CN
- China
- Prior art keywords
- data
- hbase
- target
- rowkey
- line unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000004044 response Effects 0.000 claims abstract description 14
- 238000013481 data capture Methods 0.000 claims description 25
- 230000005540 biological transmission Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 2
- 230000015654 memory Effects 0.000 description 12
- 238000012545 processing Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000005291 magnetic effect Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/221—Column-oriented storage; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2468—Fuzzy queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Automation & Control Theory (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a data acquisition method, a data acquisition device, data acquisition equipment and a storage medium based on HBase. The method comprises the following steps: determining a target query keyword in response to a data query request of a target user; matching the target query keyword with pre-established index configuration information, and determining a target row key Rowkey matched with the target query keyword; wherein at least one row key Rowkey is indexed by one query keyword in the index configuration information; and acquiring query data from a data table of the HBase database according to the target row key Rowkey. By adopting the scheme of the embodiment, the speed of HBase fuzzy search can be increased, the speed is fastest when a large amount of data is processed, a plurality of query keywords are supported to jointly search row keys, and data acquisition of a plurality of query condition dimensions is realized.
Description
Technical field
The present embodiments relate to database technical field more particularly to a kind of data capture methods based on HBase, dress
It sets, equipment and storage medium.
Background technique
With the rapid development of information technology, data start explosive increase, have extensive data source and efficiently it is fast
The data of speed generate, and distributed storage system is come into being in this context.
HBase is a high reliability, high-performance, towards column, telescopic distributed memory system, utilizes HBase skill
Art can erect large-scale structure storage cluster on cheap PC Server.HBase be one it is distributed, towards column
PostgreSQL database, HBase are different from general relational database, it is the data for being suitable for unstructured data storage
Library.HBase provides efficient write performance and flexible storage mode as column storing data library, for data storage.
However, HBase itself only provides the inquiry based on line unit and full table scan, and line unit RowKey Index List one, it is right
It is difficult in the inquiry of many condition dimension, while single line unit RowKey index is likely to that data query is caused to become full table scan,
It is poor to the support of associated data or complex conditions when inquiring mass data.
Summary of the invention
A kind of data capture method based on HBase, device, equipment and storage medium are provided in the embodiment of the present invention,
Accelerate fuzzy query speed in HBase to realize, and supports the data acquisition of many condition dimension.
In a first aspect, providing a kind of data capture method based on HBase in the embodiment of the present invention, comprising: in response to
The data inquiry request of target user determines target query keyword；By the target query keyword and the rope being pre-created
Draw configuration information to be matched, the determining target line unit Rowkey with the target query Keywords matching；The wherein index
A searching keyword indexes at least one line unit Rowkey in configuration information；According to the target line unit Rowkey, from HBase
Inquiry data are obtained in the tables of data of database.
Second aspect additionally provides a kind of data acquisition facility based on HBase in the embodiment of the present invention, comprising: inquiry
Ask respond module determines target query keyword for the data inquiry request in response to target user；Searching keyword
With module, for matching the target query keyword with the index configurations information being pre-created, determination is looked into target
Ask the target line unit Rowkey of Keywords matching；Wherein a searching keyword indexes at least one in the index configurations information
Line unit Rowkey；Data acquisition module is inquired, is used for according to the target line unit Rowkey, from the tables of data of HBase database
Data are inquired in middle acquisition.
The third aspect additionally provides a kind of electronic equipment in the embodiment of the present invention, comprising:
One or more processors；
Storage device, for storing one or more programs；
When one or more of programs are executed by one or more of processors, so that one or more of processing
Device realizes the data capture method based on HBase as provided in any embodiment of that present invention.
Fourth aspect additionally provides a kind of computer readable storage medium in the embodiment of the present invention, is stored thereon with calculating
Machine program realizes the data acquisition based on HBase as provided in any embodiment of that present invention when the program is executed by processor
Method.
A kind of data acquisition scheme based on HBase is provided in the embodiment of the present invention, in the data query of response user
It, can be according to searching keyword first from the index configurations information being pre-created when request obtains data from HBase data base querying
The associated line unit Rowkey of searching keyword is found, may be implemented to carry out line unit using multiple queries keyword using aforesaid way
Rowkey determines matching, subsequent to be obtained according to the tables of data of the slave HBase database of matched line unit Rowkey batch
Inquire data.Using this embodiment scheme due to using the index configurations information matches line unit Rowkey, Jin Erji that are pre-created
It is efficiently inquired in line unit Rowkey in HBase database, therefore the speed that HBase is searched for generally can be accelerated, make locating
Speed is reached most fast when managing mass data, and supports multiple queries keyword to find line unit Rowkey jointly, is realized more
The data acquisition of a querying condition dimension.
Foregoing invention content is only the general introduction of technical solution of the present invention, in order to better understand technology hand of the invention
Section, and can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage energy
It is enough clearer and more comprehensible, the followings are specific embodiments of the present invention.
Detailed description of the invention
By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, of the invention other
Feature, objects and advantages will become more apparent upon.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as pair
Limitation of the invention.In the accompanying drawings:
Fig. 1 is a kind of flow chart of the data capture method based on HBase provided in the embodiment of the present invention；
A kind of interaction schematic diagram of the fuzzy query data in HBase provided in the embodiment of the present invention when Fig. 2；
Fig. 3 is the flow chart of another data capture method based on HBase provided in the embodiment of the present invention；
Fig. 4 is a kind of structural schematic diagram of the data acquisition facility based on HBase provided in the embodiment of the present invention；
Fig. 5 is the structural schematic diagram of a kind of electronic equipment provided in the embodiment of the present invention.
Specific embodiment
The present invention is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining the present invention rather than limiting the invention.It also should be noted that in order to just
Only the parts related to the present invention are shown in description, attached drawing rather than entire infrastructure.
It should be mentioned that some exemplary embodiments are described as before exemplary embodiment is discussed in greater detail
The processing or method described as flow chart.Although operations (or step) are described as the processing of sequence by flow chart,
Many of these operations can be implemented concurrently, concomitantly or simultaneously.In addition, the sequence of operations can be pacified again
Row.The processing can be terminated when its operations are completed, it is also possible to have the additional step being not included in attached drawing.Institute
Stating processing can correspond to method, function, regulation, subroutine, subprogram etc..
The phase of HBase database is described in detail below in technical solution in embodiment for a better understanding of the present invention
Hold inside the Pass.On the storage organization of HBase, HBase is ranked up according to lexicographic order, and line unit RowKey is directly as therein
Level-one index, provides the foundation for its good readwrite performance.HBase advantage has very much, such as semi-structured or unstructured
Data, High Availabitity and mass data and very big moment writing require nothing more than final consistency etc..But HBase is also the same
There are many disadvantages, for example the intrinsic limitation of single RowKey determines that it can not effectively support multi-condition inquiry, with
And it is not suitable for large area scanning inquiry data.
Fig. 1 is a kind of flow chart of the data capture method based on HBase provided in the embodiment of the present invention.The present invention is real
Apply the case where example is applicable to the fuzzy query data in HBase.The data capture method based on HBase can be by being based on
The data acquisition facility of HBase executes, which can be realized by the way of software and/or hardware, which can be with
It is configured in any electronic equipment with network communicating function.As shown in Figure 1, provided in the embodiment of the present invention based on
The data capture method of HBase, specifically includes the following steps:
S110, the data inquiry request in response to target user, determine target query keyword.
In the present embodiment, target user refers to when based on HBase inquiry data, can send data to electronic equipment
The user of inquiry request.Electronic equipment can receive and respond the data inquiry request of target user's transmission, and determine that data are looked into
Ask the target query keyword carried in request.One or more inquiries in target query keyword comprising user's input are crucial
Word.For example, when user wants that inquiry is about the relevant data such as " Google ", " map ", " social activity " in HBase database, user
" Google ", " map " and " social activity " can be inputted on the terminal device, by " Google ", " map " and " social activity " three keywords
Target query keyword is formed, and is sent to electronic equipment in the form of data inquiry request.
S120, target query keyword is matched with the index configurations information being pre-created, determining and target query
The target line unit Rowkey of Keywords matching；Wherein a searching keyword indexes at least one line unit in index configurations information
Rowkey。
In the present embodiment, be stored in advance in index configurations information each searching keyword and each line unit Rowkey it
Between mapping relations, and each searching keyword indexes at least one line unit Rowkey in index configurations information, that is, exists
In index configurations information one searching keyword can there are mapping relations with one or more line unit Rowkey.It is inquired when one
When the multiple line unit Rowkey of keyword index, by a searching keyword, may matching obtain multiple line unit Rowkey.
It in the present embodiment, can be according to include in target query keyword one after determining target query keyword
A or multiple queries keyword is searched in the index configurations information being pre-created.If target query keyword can be found
In include searching keyword, then by the searching keyword for including in target query keyword index one or more rows be good for
Rowkey, the target line unit Rowkey obtained as matching.
It is advantageous in that using aforesaid way, is obtained in the data inquiry request of response user from HBase data base querying
When data, the row that multiple queries keyword while index can be obtained in index configurations information is good for Rowkey, in this case
May be implemented to search line unit Rowkey using multiple queries keyword, it is subsequent can according to the batch line unit Rowkey of lookup from
The tables of data of HBase database obtains inquiry data.
In the present embodiment, optionally, not only comprising stored in HBase database in index configurations information
The index relative of the line unit Rowkey of KeyValue data (being denoted as storage KeyValue data), but also may include new every time
Increase to the line unit Rowkey for the single KeyValue data (being denoted as increment KeyValue data) that HBase database is stored
Index relative.
S130, foundation target line unit Rowkey, obtain inquiry data from the tables of data of HBase database.
In the present embodiment, HBase is orderly stored based on the strong Rowkey of row, can be very high using Rowkey in inquiry
Effect carries out data query according to line unit Rowkey.Target line unit Rowkey is being obtained, it can be based on determining target line unit
The each line unit Rowkey for including in Rowkey is inquired in the tables of data of HBase database and is determined that line unit Rowkey is corresponding
Data, and using the corresponding data of line unit Rowkey as final inquiry data.Wherein, the built-in two storage table (ROOT of HBase
And META) carry out the storage of area distribution and region details.It, can be according to line unit from Rowkey in data retrieval process
The storage region of data is inquired in this two built-in tables, to facilitate acquisition inquiry data.
In the present embodiment, a kind of interaction of the fuzzy query data in HBase provided in the embodiment of the present invention when Fig. 2
Schematic diagram.Referring to fig. 2, optionally, HBase database can externally provide Rowkey batch query interface, realize simultaneously to more
A Rowkey is inquired.After determining target line unit Rowkey, the Rowkey batch query of calling HBase database can be passed through
Interface, instruction carry out data query based on target line unit Rowkey in HBase database.In turn, it can receive from HBase number
According to the corresponding inquiry data of target line unit Rowkey that inquiry is read and returned in the tables of data in library.
In the present embodiment, optionally, the type of the corresponding inquiry data of the target line unit Rowkey of return can be
Result array, electronic equipment can be carried out at this time with the corresponding inquiry data of the target line unit Rowkey of Result array type
Traversal, the data set merging for being encapsulated into JSON type return to target user.
A kind of data acquisition scheme based on HBase is provided in the embodiment of the present invention, in the data query of response user
It, can be according to searching keyword first from the index configurations information being pre-created when request obtains data from HBase data base querying
The associated line unit Rowkey of searching keyword is found, may be implemented to carry out line unit using multiple queries keyword using aforesaid way
Rowkey determines matching, subsequent to be obtained according to the tables of data of the slave HBase database of matched line unit Rowkey batch
Inquire data.Using this embodiment scheme due to using the index configurations information matches line unit Rowkey, Jin Erji that are pre-created
It is efficiently inquired in line unit Rowkey in HBase database, therefore the speed that HBase is searched for generally can be accelerated, make locating
Speed is reached most fast when managing mass data, and supports multiple queries keyword to find line unit Rowkey jointly, is realized more
The data acquisition of a querying condition dimension.
Fig. 3 is the flow chart of another data capture method based on HBase provided in the embodiment of the present invention.The present invention
Embodiment optimizes on the basis of the above embodiments, the embodiment of the present invention can in said one or multiple embodiments
Each optinal plan combines.As shown in figure 3, the data capture method based on HBase provided in the embodiment of the present invention, specific to wrap
Include following steps:
S310, the data inquiry request in response to target user, determine target query keyword.
S320, foundation target query keyword, instruct to ElasticSearch collection pocket transmission target query, with instruction
ElasticSearch cluster searches target query keyword based on index configurations information.
In the present embodiment, ElasticSearch is the search server based on Lucene, is capable of providing distribution
The full-text search engine of multi-user capability.The search speed of ElasticSearch is fast, facilitates foundation to index and full text is supported to examine
Rope.Distributed real-time files storage may be implemented in ElasticSearch, and each data field is indexed, and makes it
It can be searched.In addition, ElasticSearch is a highly scalable search engine, it can extend to up to a hundred services
Device handles structuring or the unstructured data of PB rank, realizes building distributed search engine.
In the present embodiment, each KeyValue data that ElasticSearch can will be stored in HBase database
Field is all indexed, and forms index configurations information, to guarantee that each searching keyword index is at least in index configurations information
One line unit Rowkey.Wherein, the searching keyword in index configurations information can be ElasticSearch to KeyValue number
Each keyword is obtained after being segmented according to field.The KeyValue data that ElasticSearch will be stored in HBase database
After field is all indexed to generate index configurations information, while index configurations information being stored.
In the present embodiment, referring to fig. 2, after determining target query keyword, electronic equipment can be according to target query
Keyword generates target query instruction, and target query instruction is sent to and electronic equipment communication connection
ElasticSearch cluster.ElasticSearch cluster can receive target query instruction, and obtain in target query instruction
The target query keyword of carrying.In turn, ElasticSearch cluster can be searched in pre-stored index configurations information
Target query keyword determines in index configurations information with the presence or absence of the target query keyword.If being deposited in index configurations information
In the target query keyword, then one or more rows of the target query keyword index are good for Rowkey as target line unit
Rowkey。
In the present embodiment, optionally, the Java API couple that electronic equipment can be provided by calling ElasticSearch
ElasticSearch initiates data query operation, to indicate that ElasticSearch cluster searches target in index configurations information
Searching keyword.
S330, it receives in the index configurations information that ElasticSearch cluster is searched and is closed with target query keyword
The target line unit Rowkey of connection.
In the present embodiment, referring to fig. 2, it is looked into when ElasticSearch cluster inquires target in index configurations information
After the strong Rowkey of one or more rows for asking keyword index, ElasticSearch cluster can be by inquire one or more
A row is good for Rowkey and returns to electronic equipment as target line unit Rowkey, and by target line unit Rowkey.Correspondingly, electronics is set
It is standby can receive in the index configurations information of ElasticSearch collection pocket transmission with the associated target line unit of target query keyword
Rowkey。
In the present embodiment, can store in HBase database the stored storage KeyValue data of history and every time
Newly increase the single increment KeyValue data of storage.Correspondingly, index configurations information is by ElasticSearch cluster pair
The storage KeyValue data and/or single increment KeyValue data of batch in HBase database carry out at inverted index
Manage the index relative of obtained line unit Rowkey.
In the present embodiment, table 1 is a kind of row's of falling rope of the simple inverted index processing provided in the embodiment of the present invention
Draw table, inverted index is described in detail below with reference to table 1.Inverted index is the one kind for realizing " word-document matrix "
Specific storage form can include the lists of documents of this word according to word quick obtaining by inverted index.Inverted index
It is mainly made of two parts: " word lexicon " and " inverted file ".Word lexicon (Lexicon): the usual rope of search engine
Drawing unit is word, and word lexicon is the string assemble being made of all words occurred in collection of document, word lexicon
Interior every index entry records some information of word itself and is directed toward the pointer of " Inverted List ".Inverted List
(PostingList): Inverted List describes the lists of documents for all documents of some word occurred and word in the document
The location information of middle appearance, every record are known as the row of falling item (Posting).According to Inverted List, you can learn which is literary
Shelves include some word.Inverted file (Inverted File): the Inverted List of all words is often sequentially stored in disk
Some file in, this file is referred to as inverted file, and inverted file is the physical file for storing inverted index.
Optionally, by taking the searching keyword in the present embodiment as an example, " word " in table 1 be can be understood as in the present embodiment
Searching keyword, " word ID " can be understood as the ID of the searching keyword in the present embodiment, in " Inverted List (DocID) "
The row of record queries keyword index is good for Rowkey.
The inverted index table of the simple inverted index of table 1 processing
| Word ID | Word | Inverted List (DocID) |
| 1 | 1、2、3、4、5 | |
| 2 | Map | 1、2、3、4、5 |
| 3 | Father | 1、2、4、5 |
| 4 | Job-hopping | 1、2、3、4、5 |
| 5 | 2、3、5 | |
| 6 | Join | 3 |
| 7 | Founder | 3、5 |
| 8 | Lars | 3 |
| 9 | It leaves | 4 |
| 10 | With | 4 |
| 11 | Wave | 4 |
| 12 | Project | 4 |
| 13 | Cancel | 4 |
| 14 | It is related | 4 |
| 15 | It is social | 5 |
| 16 | Website | 5 |
It in the present embodiment, not only may include ElasticSearch cluster in index configurations information to HBase database
The storage KeyValue data of middle batch carry out the line unit Rowkey information that inverted index is handled, but also may include
ElasticSearch cluster carries out at inverted index each increment KeyValue data newly increased in HBase database
Manage obtained line unit Rowkey information.
ElasticSearch cluster is directed in terms of storage KeyValue data and increment KeyValue data two below
Inverted index is carried out to KeyValue data in HBase database, the process for generating index configurations information is described in detail.
In a kind of optional way of the present embodiment, referring to fig. 2, index configurations information by ElasticSearch cluster according to
According to storage KeyValue data stored from the HBase database being written in batches in HBase database, inverted index is carried out
Processing obtains.Optionally, the storage KeyValue data in reading HBase database, by the storage KeyValue number of batch
According to write-in ElasticSearch cluster, ElasticSearch cluster can create index in batches, obtain index configurations information, and
It is saved.
In the present embodiment, HFile is the storage format of KeyValue data in HBase database, and HFile is
The binary format file of Hadoop, actually StoreFile can be grade packaged, the i.e. StoreFile that done light weight to HFile
Bottom is exactly HFile.Optionally, in the present embodiment, when reading the KeyValue data in HBase database, Ke Yizhi
The data block for reading each HFile file in HBase database is connect, and traverses the KeyValue data read in all data blocks,
Rather than the query interface API on the upper layer HBase is called to be successively read.It is advantageous in that using above-mentioned, bypasses HBase top layer
API, data reading speed can be greatly improved by directly parsing HFile.
In the present embodiment, HFile file is mainly made of six parts in HBase database, is respectively as follows: Data (number
According to block), Meta (meta data block), File Info, Data Index (data block index), Meta Index (meta data block rope
Draw) and Trailer.The specific process for reading KeyValue data in HBase database are as follows: HFile in creation HBase database
The read path of file；FSdataInputStream is created, the size of HFile file in HBase database is obtained；Creation
FixedFileTrailer reads tail blocks of files；Create HFileReaderV2 object.In turn, pass through HFileReaderV2
GetScanner () in object creates scanner Scanner, and is read in HBase database using scanner Scanner
HFile file data blocks.Finally, traversing the KeyValue data in all data blocks of reading again.
It is referring to fig. 2, new when one is written in HBase database in another optional way of the present embodiment
When KeyValue data, index configurations information is also single according to being written through message queue Kafka by ElasticSearch cluster
KeyValue data carry out inverted index update.
In the present embodiment, when a new increment KeyValue data are written in HBase database, this is new
Increment KeyValue data be stored in HBase database while, the new KeyValue data of this are also sent to message team
It arranges Kafka and carries out asynchronous process.This increment KeyValue data are passed through ElasticSearch cluster by Kafka consumer
Inverted index is written in Java API, to generate index configurations information.
S340, foundation target line unit Rowkey, obtain inquiry data from the tables of data of HBase database.
A kind of data acquisition scheme based on HBase is provided in the embodiment of the present invention, can use ElasticSearch
Cluster carries out inverted index to KeyValue data in HBase database and generates index configurations information, is pre-created due to using
Index configurations information matches line unit Rowkey, and then efficiently inquired in HBase database based on line unit Rowkey, therefore
It can accelerate the speed that HBase is searched for generally, make that speed is reached most fast when handling mass data, and support multiple queries
Keyword finds line unit Rowkey jointly, realizes the data acquisition of multiple queries conditional dimension.
Fig. 4 is a kind of structural schematic diagram of the data acquisition facility based on HBase provided in the embodiment of the present invention.This hair
Bright embodiment is applicable to the case where fuzzy query data in HBase.The device can be by the way of software and/or hardware
Realize, should data acquisition facility based on HBase can be configured in any electronic equipment with network communicating function.Such as
Shown in Fig. 4, the data acquisition facility based on HBase provided in the embodiment of the present invention be can specifically include: inquiry request response
Module 410, searching keyword matching module 420 and inquiry data acquisition module 430.Wherein:
Inquiry request respond module 410 determines target query key for the data inquiry request in response to target user
Word；
Searching keyword matching module 420, for believing the target query keyword with the index configurations being pre-created
Breath is matched, the determining target line unit Rowkey with the target query Keywords matching；The wherein index configurations information
In a searching keyword index at least one line unit Rowkey；
Data acquisition module 430 is inquired, is used for according to the target line unit Rowkey, from the tables of data of HBase database
Data are inquired in middle acquisition.
On the basis of the technical solution of above-described embodiment, optionally, searching keyword matching mould 420 includes:
Query statement transmitting element is used for according to the target query keyword, to ElasticSearch collection pocket transmission mesh
Inquiry instruction is marked, is closed with indicating that the ElasticSearch cluster is based on target query described in the index configurations information searching
Keyword；
Line unit information acquisition unit, the index configurations searched for receiving the ElasticSearch cluster
In information with the associated target line unit Rowkey of the target query keyword.
On the basis of the technical solution of above-described embodiment, optionally, the index configurations information is by ElasticSearch
Cluster carries out inverted index and handles to obtain according to the KeyValue data from HBase database batch write-in.
On the basis of the technical solution of above-described embodiment, optionally, the KeyValue data of the batch write-in are logical
The data block for directly reading each HFile file in HBase database is crossed, and traverses the KeyValue in all data blocks of acquisition
It obtains.
On the basis of the technical solution of above-described embodiment, optionally, new when one is written in HBase database
When KeyValue data, the index configurations information through message queue Kafka also by ElasticSearch cluster according to being written
Single KeyValue data, carry out inverted index update.
It on the basis of the technical solution of above-described embodiment, optionally, include HBase data in the index configurations information
The line unit Rowkey information of stored KeyValue data and/or the KeyValue data newly increased in library.
On the basis of the technical solution of above-described embodiment, optionally, inquiry data acquisition module 430 includes:
Interface calling module, the Rowkey batch query interface for calling HBase database to provide, is existed with instruction
HBase database is based on target line unit Rowkey and carries out data query；
Inquire data capture unit, for receives in the tables of data of HBase database carry out data query and return
Inquiry data corresponding with target line unit Rowkey.
Aforementioned present invention, which can be performed, in data acquisition facility provided in the embodiment of the present invention based on HBase arbitrarily implements
Data capture method based on HBase provided in example, has and executes the corresponding function of data capture method based on HBase
Can and beneficial effect, the not technical detail of detailed description in the above-described embodiments, reference can be made to being mentioned in the application any embodiment
The data capture method based on HBase supplied.
Fig. 5 is the structural schematic diagram of a kind of electronic equipment provided in the embodiment of the present invention.Structure as shown in Figure 5, this hair
The electronic equipment provided in bright embodiment includes: one or more processors 510 and storage device 520；In the electronic equipment
Processor 510 can be one or more, in Fig. 5 by taking a processor 510 as an example；Storage device 520 for store one or
Multiple programs；One or more of programs are executed by one or more of processors 510, so that one or more of places
Manage data capture method based on HBase of the realization of device 510 as described in any one of embodiment of the present invention.
The electronic equipment can also include: input unit 530 and output device 540.
Processor 510, storage device 520, input unit 530 and output device 540 in the electronic equipment can pass through
Bus or other modes connect, in Fig. 5 for being connected by bus.
Storage device 520 in the electronic equipment is used as a kind of computer readable storage medium, can be used for storing one or
Multiple programs, described program can be software program, computer executable program and module, as mentioned in the embodiment of the present invention
Corresponding program instruction/the module of the data capture method based on HBase supplied is (for example, the attached number shown in Fig. 4 based on HBase
According to the module in acquisition methods device, comprising: inquiry request respond module 410, searching keyword matching module 420 and inquiry number
According to acquisition module 430).Software program, instruction and the module that processor 510 is stored in storage device 520 by operation, from
And execute the various function application and data processing of electronic equipment, that is, realize the number based on HBase in above method embodiment
According to acquisition methods.
Storage device 520 may include storing program area and storage data area, wherein storing program area can store operation system
Application program needed for system, at least one function；Storage data area, which can be stored, uses created data according to electronic equipment
Deng.In addition, storage device 520 may include high-speed random access memory, it can also include nonvolatile memory, such as extremely
A few disk memory, flush memory device or other non-volatile solid state memory parts.In some instances, storage device
520 can further comprise the memory remotely located relative to processor 510, these remote memories can pass through network connection
To equipment.The example of above-mentioned network includes but is not limited to internet, intranet, local area network, mobile radio communication and combinations thereof.
Input unit 530 can be used for receiving the number or character information of input, and generates and set with the user of electronic equipment
It sets and the related key signals of function control inputs.Output device 540 may include that display screen etc. shows equipment.
Also, when one or more included program of above-mentioned electronic equipment is by one or more of processors 510
When execution, program is proceeded as follows:
In response to the data inquiry request of target user, target query keyword is determined；
The target query keyword is matched with the index configurations information being pre-created, determination is looked into the target
Ask the target line unit Rowkey of Keywords matching；Wherein a searching keyword indexes at least one in the index configurations information
Line unit Rowkey；
According to the target line unit Rowkey, inquiry data are obtained from the tables of data of HBase database.
Certainly, it will be understood by those skilled in the art that when one or more included program of above-mentioned electronic equipment is by institute
State one or more processor 510 execute when, program can also carry out provided in any embodiment of that present invention based on
Relevant operation in the data capture method of HBase.
A kind of computer readable storage medium is provided in the embodiment of the present invention, is stored thereon with computer program, the journey
For executing the data capture method based on HBase when sequence is executed by processor, this method comprises:
In response to the data inquiry request of target user, target query keyword is determined；
The target query keyword is matched with the index configurations information being pre-created, determination is looked into the target
Ask the target line unit Rowkey of Keywords matching；Wherein a searching keyword indexes at least one in the index configurations information
Line unit Rowkey；
According to the target line unit Rowkey, inquiry data are obtained from the tables of data of HBase database.
Optionally, it can be also used for executing base provided in any embodiment of that present invention when which is executed by processor
In the data capture method of HBase.
The computer storage medium of the embodiment of the present invention, can be using any of one or more computer-readable media
Combination.Computer-readable medium can be computer-readable signal media or computer readable storage medium.It is computer-readable
Storage medium for example may be-but not limited to-the system of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, device or
Device, or any above combination.The more specific example (non exhaustive list) of computer readable storage medium includes: tool
There are electrical connection, the portable computer diskette, hard disk, random access memory (Random Access of one or more conducting wires
Memory, RAM), read-only memory (Read Only Memory, ROM), erasable programmable read only memory (Erasable
Programmable Read Only Memory, EPROM), flash memory, optical fiber, portable CD-ROM, light storage device, magnetic storage
Device or above-mentioned any appropriate combination.Computer readable storage medium, which can be, any includes or storage program has
Shape medium, the program can be commanded execution system, device or device use or in connection.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In: electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.
The program code for including on computer-readable medium can transmit with any suitable medium, including but not limited to:
Wirelessly, electric wire, optical cable, radio frequency (Radio Frequency, RF) etc. or above-mentioned any appropriate combination.
The computer for executing operation of the present invention can be write with one or more programming languages or combinations thereof
Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++,
It further include conventional procedural programming language-such as " C " language or similar programming language.Program code can be with
It fully executes, partly execute on the user computer on the user computer, being executed as an independent software package, portion
Divide and partially executes or executed on a remote computer or server completely on the remote computer on the user computer.?
Be related in the situation of remote computer, remote computer can pass through the network of any kind --- including local area network (LAN) or
Wide area network (WAN) --- it is connected to subscriber computer, or, it may be connected to outer computer (such as utilize Internet service
Provider is connected by internet).
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or spy described in conjunction with this embodiment or example
Point is included at least one embodiment or example of the invention.In the present specification, schematic expression of the above terms are not
Centainly refer to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be any
One or more embodiment or examples in can be combined in any suitable manner.
Note that the above is only a better embodiment of the present invention and the applied technical principle.It will be appreciated by those skilled in the art that
The invention is not limited to the specific embodiments described herein, be able to carry out for a person skilled in the art it is various it is apparent variation,
It readjusts and substitutes without departing from protection scope of the present invention.Therefore, although being carried out by above embodiments to the present invention
It is described in further detail, but the present invention is not limited to the above embodiments only, without departing from the inventive concept, also
It may include more other equivalent embodiments, and the scope of the invention is determined by the scope of the appended claims.
Claims (16)
1. a kind of data capture method based on HBase characterized by comprising
In response to the data inquiry request of target user, target query keyword is determined；
The target query keyword is matched with the index configurations information being pre-created, it is determining to be closed with the target query
The matched target line unit Rowkey of keyword；Wherein a searching keyword indexes at least one line unit in the index configurations information
Rowkey；
According to the target line unit Rowkey, inquiry data are obtained from the tables of data of HBase database.
2. the method according to claim 1, wherein by the target query keyword and the index being pre-created
Configuration information is matched, the determining target line unit Rowkey with the target query Keywords matching, comprising:
It according to the target query keyword, is instructed to ElasticSearch collection pocket transmission target query, described in instruction
ElasticSearch cluster is based on target query keyword described in the index configurations information searching；
Receive in the index configurations information that the ElasticSearch cluster is searched with the target query keyword
Associated target line unit Rowkey.
3. method according to claim 1 or 2, which is characterized in that the index configurations information is by ElasticSearch collection
Group carries out inverted index and handles to obtain according to the KeyValue data from HBase database batch write-in.
4. according to the method described in claim 3, it is characterized in that, the KeyValue data of batch write-in are by direct
The data block of each HFile file in HBase database is read, and the KeyValue traversed in all data blocks of acquisition is obtained.
5. according to the method described in claim 3, it is characterized in that, when a new KeyValue is written in HBase database
When data, the index configurations information is also single according to being written through message queue Kafka by ElasticSearch cluster
KeyValue data carry out inverted index update.
6. method according to claim 1 or 2, which is characterized in that include HBase database in the index configurations information
In the line unit Rowkey information of stored KeyValue data and/or the KeyValue data newly increased.
7. the method according to claim 1, wherein according to the target line unit Rowkey, from HBase database
Tables of data in obtain inquiry data, comprising:
The Rowkey batch query interface for calling HBase database to provide is based on target line unit in HBase database with instruction
Rowkey carries out data query；
Receive the inquiry corresponding with target line unit Rowkey for carrying out data query in the tables of data of HBase database and returning
Data.
8. a kind of data acquisition facility based on HBase characterized by comprising
Inquiry request respond module determines target query keyword for the data inquiry request in response to target user；
A searching keyword matching module, for carrying out the target query keyword and the index configurations information that is pre-created
Match, the determining target line unit Rowkey with the target query Keywords matching；Wherein in the index configurations information one look into
Ask at least one line unit of keyword index Rowkey；
Data acquisition module is inquired, for obtaining and looking into from the tables of data of HBase database according to the target line unit Rowkey
Ask data.
9. device according to claim 8, which is characterized in that searching keyword matches mould and includes:
Query statement transmitting element, for being looked into ElasticSearch collection pocket transmission target according to the target query keyword
Instruction is ask, to indicate that the ElasticSearch cluster is based on target query keyword described in the index configurations information searching；
Line unit information acquisition unit, the index configurations information searched for receiving the ElasticSearch cluster
In with the associated target line unit Rowkey of the target query keyword.
10. device according to claim 8 or claim 9, which is characterized in that the index configurations information is by ElasticSearch
Cluster carries out inverted index and handles to obtain according to the KeyValue data from HBase database batch write-in.
11. device according to claim 10, which is characterized in that the KeyValue data of the batch write-in are by straight
The data block for reading each HFile file in HBase database is connect, and the KeyValue traversed in all data blocks of acquisition is obtained
It arrives.
12. device according to claim 10, which is characterized in that new when one is written in HBase database
When KeyValue data, the index configurations information through message queue Kafka also by ElasticSearch cluster according to being written
Single KeyValue data, carry out inverted index update.
13. device according to claim 8 or claim 9, which is characterized in that include HBase data in the index configurations information
The line unit Rowkey information of stored KeyValue data and/or the KeyValue data newly increased in library.
14. device according to claim 8, which is characterized in that inquiring data acquisition module includes:
Interface calling module, the Rowkey batch query interface for calling HBase database to provide, to indicate in HBase number
Target line unit Rowkey, which is based on, according to library carries out data query；
Inquire data capture unit, for receives in the tables of data of HBase database carry out data query and return and mesh
Mark the corresponding inquiry data of line unit Rowkey.
15. a kind of electronic equipment characterized by comprising
One or more processors；
Storage device, for storing one or more programs；
When one or more of programs are executed by one or more of processors, so that one or more of processors are real
Any data capture method based on HBase in existing claim 1-7.
16. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
Any data capture method based on HBase in claim 1-7 is realized when execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910625423.5A CN110347722A (en) | 2019-07-11 | 2019-07-11 | Data acquisition method, device, equipment and storage medium based on HBase |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910625423.5A CN110347722A (en) | 2019-07-11 | 2019-07-11 | Data acquisition method, device, equipment and storage medium based on HBase |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110347722A true CN110347722A (en) | 2019-10-18 |
Family
ID=68175132
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910625423.5A Pending CN110347722A (en) | 2019-07-11 | 2019-07-11 | Data acquisition method, device, equipment and storage medium based on HBase |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110347722A (en) |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110502607A (en) * | 2019-06-26 | 2019-11-26 | 中电万维信息技术有限责任公司 | A kind of electronic medical record system, the method and server for inquiring electronic health record |
| CN110866085A (en) * | 2019-11-04 | 2020-03-06 | 北京百分点信息科技有限公司 | Data feedback method and device |
| CN111078731A (en) * | 2019-11-25 | 2020-04-28 | 国网冀北电力有限公司 | Hbase-based power grid operation data collaborative query method and device and storage medium |
| CN111125119A (en) * | 2019-12-30 | 2020-05-08 | 中科星图股份有限公司 | HBase-based spatio-temporal data storage and indexing method |
| CN111488386A (en) * | 2020-04-14 | 2020-08-04 | 北京易数科技有限公司 | Data query method and device |
| CN111680018A (en) * | 2020-04-26 | 2020-09-18 | 泰康保险集团股份有限公司 | Method, device and equipment for responding to request and readable storage medium |
| CN111858831A (en) * | 2020-07-07 | 2020-10-30 | 杭州安恒信息技术股份有限公司 | Elastic search indexing method and device based on APT analysis and computer equipment |
| CN112069175A (en) * | 2020-08-25 | 2020-12-11 | 北京五八信息技术有限公司 | Data query method and device and electronic equipment |
| CN112162982A (en) * | 2020-09-16 | 2021-01-01 | 中国建设银行股份有限公司 | Data query method, device, equipment and medium |
| CN112463800A (en) * | 2020-12-11 | 2021-03-09 | 微医云(杭州)控股有限公司 | Data reading method and device, server and storage medium |
| CN112686551A (en) * | 2020-12-31 | 2021-04-20 | 北京千方科技股份有限公司 | Flight cost index obtaining method and device, electronic equipment and storage medium |
| CN112699149A (en) * | 2020-12-31 | 2021-04-23 | 青岛海尔科技有限公司 | Target data acquisition method and device, storage medium and electronic device |
| CN112818195A (en) * | 2019-11-15 | 2021-05-18 | 腾讯科技（深圳）有限公司 | Data acquisition method, device and system and computer storage medium |
| CN112835905A (en) * | 2021-02-05 | 2021-05-25 | 上海达梦数据库有限公司 | Indexing method, device, equipment and storage medium for array type column |
| CN113127488A (en) * | 2021-04-20 | 2021-07-16 | 粒子文化科技集团(杭州)股份有限公司 | Data query method, device, electronic device and storage medium |
| CN113190548A (en) * | 2020-12-24 | 2021-07-30 | 武汉烽火众智数字技术有限责任公司 | HBASE-based archive library design method |
| CN113297273A (en) * | 2021-06-09 | 2021-08-24 | 北京百度网讯科技有限公司 | Method and device for querying metadata and electronic equipment |
| CN113448957A (en) * | 2020-03-24 | 2021-09-28 | 北京沃东天骏信息技术有限公司 | Data query method and device |
| CN113626473A (en) * | 2021-08-17 | 2021-11-09 | 北京京东振世信息技术有限公司 | Data information query method and device |
| CN113672963A (en) * | 2021-08-30 | 2021-11-19 | 国家计算机网络与信息安全管理中心 | Matching method and device based on rainbow table, storage medium and electronic equipment |
| CN114785750A (en) * | 2022-04-06 | 2022-07-22 | 厦门立林科技有限公司 | Access control message reminding method, system and equipment based on multi-community application |
| CN116108023A (en) * | 2023-04-12 | 2023-05-12 | 北京首信科技股份有限公司 | Data retrieval method and device for HBase database |
| CN116401259A (en) * | 2023-06-08 | 2023-07-07 | 北京江融信科技有限公司 | Automatic pre-creation index method and system for elastic search database |
| WO2023143095A1 (en) * | 2022-01-25 | 2023-08-03 | Zhejiang Dahua Technology Co., Ltd. | Method and system for data query |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104573022A (en) * | 2015-01-12 | 2015-04-29 | 浪潮软件股份有限公司 | Data query method and device for HBase |
| CN104731945A (en) * | 2015-03-31 | 2015-06-24 | 浪潮集团有限公司 | Full-text searching method and device based on HBase |
| CN105320746A (en) * | 2015-09-25 | 2016-02-10 | 北京北信源软件股份有限公司 | Big data based index acquisition method and system |
| CN105589910A (en) * | 2014-12-31 | 2016-05-18 | 中国银联股份有限公司 | HBase (Hadoop Database)-based mass transaction data retrieving method and system |
| CN106326381A (en) * | 2016-08-16 | 2017-01-11 | 梁猛 | HBase data retrieval method based on MapDB construction |
| CN106326429A (en) * | 2016-08-25 | 2017-01-11 | 武汉光谷信息技术股份有限公司 | Hbase second-level query scheme based on solr |
| CN106682139A (en) * | 2016-12-19 | 2017-05-17 | 深圳盒子支付信息技术有限公司 | Method and system for achieving HBase multi-condition query based on Solr |
| CN107103064A (en) * | 2017-04-17 | 2017-08-29 | 北京五八信息技术有限公司 | Data statistical approach and device |
| CN109271437A (en) * | 2018-09-27 | 2019-01-25 | 智庭（北京）智能科技有限公司 | A kind of Query method in real time of magnanimity rent information |
| CN109284351A (en) * | 2018-08-14 | 2019-01-29 | 青海大学 | A kind of data query method based on HBase database |
| CN109669925A (en) * | 2018-11-21 | 2019-04-23 | 北京市天元网络技术股份有限公司 | The management method and device of unstructured data |
-
2019
- 2019-07-11 CN CN201910625423.5A patent/CN110347722A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105589910A (en) * | 2014-12-31 | 2016-05-18 | 中国银联股份有限公司 | HBase (Hadoop Database)-based mass transaction data retrieving method and system |
| CN104573022A (en) * | 2015-01-12 | 2015-04-29 | 浪潮软件股份有限公司 | Data query method and device for HBase |
| CN104731945A (en) * | 2015-03-31 | 2015-06-24 | 浪潮集团有限公司 | Full-text searching method and device based on HBase |
| CN105320746A (en) * | 2015-09-25 | 2016-02-10 | 北京北信源软件股份有限公司 | Big data based index acquisition method and system |
| CN106326381A (en) * | 2016-08-16 | 2017-01-11 | 梁猛 | HBase data retrieval method based on MapDB construction |
| CN106326429A (en) * | 2016-08-25 | 2017-01-11 | 武汉光谷信息技术股份有限公司 | Hbase second-level query scheme based on solr |
| CN106682139A (en) * | 2016-12-19 | 2017-05-17 | 深圳盒子支付信息技术有限公司 | Method and system for achieving HBase multi-condition query based on Solr |
| CN107103064A (en) * | 2017-04-17 | 2017-08-29 | 北京五八信息技术有限公司 | Data statistical approach and device |
| CN109284351A (en) * | 2018-08-14 | 2019-01-29 | 青海大学 | A kind of data query method based on HBase database |
| CN109271437A (en) * | 2018-09-27 | 2019-01-25 | 智庭（北京）智能科技有限公司 | A kind of Query method in real time of magnanimity rent information |
| CN109669925A (en) * | 2018-11-21 | 2019-04-23 | 北京市天元网络技术股份有限公司 | The management method and device of unstructured data |
Cited By (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110502607A (en) * | 2019-06-26 | 2019-11-26 | 中电万维信息技术有限责任公司 | A kind of electronic medical record system, the method and server for inquiring electronic health record |
| CN110866085A (en) * | 2019-11-04 | 2020-03-06 | 北京百分点信息科技有限公司 | Data feedback method and device |
| CN112818195B (en) * | 2019-11-15 | 2023-12-26 | 腾讯科技（深圳）有限公司 | Data acquisition method, device and system and computer storage medium |
| CN112818195A (en) * | 2019-11-15 | 2021-05-18 | 腾讯科技（深圳）有限公司 | Data acquisition method, device and system and computer storage medium |
| CN111078731A (en) * | 2019-11-25 | 2020-04-28 | 国网冀北电力有限公司 | Hbase-based power grid operation data collaborative query method and device and storage medium |
| CN111125119A (en) * | 2019-12-30 | 2020-05-08 | 中科星图股份有限公司 | HBase-based spatio-temporal data storage and indexing method |
| CN113448957A (en) * | 2020-03-24 | 2021-09-28 | 北京沃东天骏信息技术有限公司 | Data query method and device |
| CN111488386A (en) * | 2020-04-14 | 2020-08-04 | 北京易数科技有限公司 | Data query method and device |
| CN111488386B (en) * | 2020-04-14 | 2023-09-29 | 北京易数科技有限公司 | Data query method and device |
| CN111680018A (en) * | 2020-04-26 | 2020-09-18 | 泰康保险集团股份有限公司 | Method, device and equipment for responding to request and readable storage medium |
| CN111858831A (en) * | 2020-07-07 | 2020-10-30 | 杭州安恒信息技术股份有限公司 | Elastic search indexing method and device based on APT analysis and computer equipment |
| CN111858831B (en) * | 2020-07-07 | 2024-04-09 | 杭州安恒信息技术股份有限公司 | APT analysis-based elastic search index method, device and computer equipment |
| CN112069175A (en) * | 2020-08-25 | 2020-12-11 | 北京五八信息技术有限公司 | Data query method and device and electronic equipment |
| CN112069175B (en) * | 2020-08-25 | 2024-03-29 | 北京五八信息技术有限公司 | Data query method and device and electronic equipment |
| CN112162982A (en) * | 2020-09-16 | 2021-01-01 | 中国建设银行股份有限公司 | Data query method, device, equipment and medium |
| CN112463800A (en) * | 2020-12-11 | 2021-03-09 | 微医云(杭州)控股有限公司 | Data reading method and device, server and storage medium |
| CN113190548A (en) * | 2020-12-24 | 2021-07-30 | 武汉烽火众智数字技术有限责任公司 | HBASE-based archive library design method |
| CN112699149A (en) * | 2020-12-31 | 2021-04-23 | 青岛海尔科技有限公司 | Target data acquisition method and device, storage medium and electronic device |
| CN112686551A (en) * | 2020-12-31 | 2021-04-20 | 北京千方科技股份有限公司 | Flight cost index obtaining method and device, electronic equipment and storage medium |
| CN112699149B (en) * | 2020-12-31 | 2023-09-19 | 青岛海尔科技有限公司 | Target data acquisition method and device, storage medium and electronic device |
| CN112835905B (en) * | 2021-02-05 | 2023-08-01 | 上海达梦数据库有限公司 | Array type column indexing method, device, equipment and storage medium |
| CN112835905A (en) * | 2021-02-05 | 2021-05-25 | 上海达梦数据库有限公司 | Indexing method, device, equipment and storage medium for array type column |
| CN113127488A (en) * | 2021-04-20 | 2021-07-16 | 粒子文化科技集团(杭州)股份有限公司 | Data query method, device, electronic device and storage medium |
| CN113297273A (en) * | 2021-06-09 | 2021-08-24 | 北京百度网讯科技有限公司 | Method and device for querying metadata and electronic equipment |
| CN113297273B (en) * | 2021-06-09 | 2024-03-01 | 北京百度网讯科技有限公司 | Method and device for inquiring metadata and electronic equipment |
| CN113626473A (en) * | 2021-08-17 | 2021-11-09 | 北京京东振世信息技术有限公司 | Data information query method and device |
| CN113672963A (en) * | 2021-08-30 | 2021-11-19 | 国家计算机网络与信息安全管理中心 | Matching method and device based on rainbow table, storage medium and electronic equipment |
| WO2023143095A1 (en) * | 2022-01-25 | 2023-08-03 | Zhejiang Dahua Technology Co., Ltd. | Method and system for data query |
| CN114785750A (en) * | 2022-04-06 | 2022-07-22 | 厦门立林科技有限公司 | Access control message reminding method, system and equipment based on multi-community application |
| CN116108023A (en) * | 2023-04-12 | 2023-05-12 | 北京首信科技股份有限公司 | Data retrieval method and device for HBase database |
| CN116401259B (en) * | 2023-06-08 | 2023-08-22 | 北京江融信科技有限公司 | Automatic pre-creation index method and system for elastic search database |
| CN116401259A (en) * | 2023-06-08 | 2023-07-07 | 北京江融信科技有限公司 | Automatic pre-creation index method and system for elastic search database |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110347722A (en) | Data acquisition method, device, equipment and storage medium based on HBase | |
| US11163957B2 (en) | Performing semantic graph search | |
| US20130282702A1 (en) | Method and system for search assistance | |
| US20120036153A1 (en) | Mobile system, search system and search result providing method for mobile search | |
| US9331971B2 (en) | Message subscription based on message aggregate characteristics | |
| US20210294865A1 (en) | System and method for updating a search index | |
| US20140331156A1 (en) | Exploring information by topic | |
| CN104239395A (en) | Method and system of searching | |
| US11232153B2 (en) | Providing query recommendations | |
| US11860870B2 (en) | High efficiency data querying | |
| US20110093489A1 (en) | Dynamic tagging | |
| CN109508361B (en) | Method and apparatus for outputting information | |
| US20130275456A1 (en) | Method and System for Content Search | |
| CN105893548A (en) | Naming method and terminal | |
| CN103353901A (en) | Orderly table data management method and system based on Hadoop distributed file system (HDFS) | |
| US10372736B2 (en) | Generating and implementing local search engines over large databases | |
| CN106716413A (en) | Unified search on a personal computing device | |
| US9465658B1 (en) | Task distribution over a heterogeneous environment through task and consumer categories | |
| Liu et al. | Finding smallest k-compact tree set for keyword queries on graphs using mapreduce | |
| CN105069175A (en) | Information retrieval method and server based on version control system | |
| US10419582B2 (en) | Processing command line templates for database queries | |
| CN112307061A (en) | Method and device for querying data | |
| CN114428834B (en) | Retrieval method, retrieval device, electronic equipment and storage medium | |
| CN112308016B (en) | Expression image acquisition method and device, electronic equipment and storage medium | |
| US9996589B2 (en) | Holistic federated ranking algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191018 |
|
| RJ01 | Rejection of invention patent application after publication |