CN108197296A - Date storage method based on Elasticsearch indexes - Google Patents
Date storage method based on Elasticsearch indexes Download PDFInfo
- Publication number
- CN108197296A CN108197296A CN201810063307.4A CN201810063307A CN108197296A CN 108197296 A CN108197296 A CN 108197296A CN 201810063307 A CN201810063307 A CN 201810063307A CN 108197296 A CN108197296 A CN 108197296A
- Authority
- CN
- China
- Prior art keywords
- data
- corresponding fragment
- index
- fragment group
- fragment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
- G06F16/137—Hash-based
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
- G06F16/134—Distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
Abstract
The invention discloses a kind of date storage method and device based on Elasticsearch indexes, this method includes：The data of acquisition are filtered with inquiry preanalysis, obtains the routing value in data；Wherein, field of the routing value for the default dimension in data；According to routing value, the corresponding fragment group of data is determined in the index；According to the ID of data, the corresponding fragment of data is determined in the corresponding fragment group of data, and data are stored in the corresponding fragment of data；The present invention passes through according to routing value, the corresponding fragment group of data is determined in the index, it can determine the fragment group of storage data, divide index using to the mode that fragment is grouped, change the storage and distribution mode of data, the expense of the consumption of performance and network during so as to reduce data search improves data search efficiency.
Description
Technical field
The present invention relates to full-text search technical field, more particularly to a kind of data based on Elasticsearch indexes are deposited
Method for storing and device.
Background technology
With the development of modern society's science and technology, in IT system now full-text search become a most common need
It asks.Wherein, bottom is based on the Elasticsearch (search server) of open source projects Lucene (full-text search engine) encapsulation,
With distribution, in real time scalability, search and data analysis capabilities, it provides the operate interface of REST API, its energy
The ability for assigning your data at the very start from project to search for, analyze and explore, out-of-the-box are current full-text search engines
First choice.
Specifically, Elasticsearch can be regarded as a Document image analysis, and Index (index) is defined
The logic storage of document and field type, each Index can include multiple Document Type (stepping type), and
Document Type are the set of Document (document).For full-text search, index is a crucial process, phase
When the storing process inside relational database, only by index operation, analysis could be carried out to data (Document) and is deposited
Storage creates inverted index, so as to which user be allowed to inquire relevant Document information.
In the prior art, when establishing index, data can be evenly distributed in respectively Elasticsearch by hash algorithm
On Shard so that when client initiates band searching request, in this case it is not apparent that the data of the request search hit are in which Shard
On, cause to need the searching request being broadcast on all Shard of all nodes, the expense of consumption and network to performance is all
It is bigger.Therefore, how by the change of the storage and distribution modes of data, the consumption of performance and network when reducing data search
Expense improves data search efficiency, is urgent problem now.
Invention content
The object of the present invention is to provide a kind of date storage method and device based on Elasticsearch indexes, with logical
The storage and distribution mode of change data is crossed, the consumption of performance and the expense of network when reducing data search improve data search effect
Rate.
In order to solve the above technical problems, the present invention provides a kind of date storage method based on Elasticsearch indexes,
Including：
Inquiry preanalysis is filtered to the data of acquisition, obtains the routing value in the data；Wherein, the routing value
Field for the default dimension in the data；
According to the routing value, the corresponding fragment group of the data is determined in the index；
According to the ID of the data, the corresponding fragment of the data is determined in the corresponding fragment group of the data, and
The data are stored in the corresponding fragment of the data.
Optionally, the quantity of the fragment group in the index is more than or equal to 2, in each fragment group in the index
The quantity of fragment is all higher than or equal to 2 and equal.
Optionally, it is described according to the routing value, the corresponding fragment group of the data is determined in the index, including：
Hash calculation is carried out to the routing value, obtains the first cryptographic Hash；
The quantity remainder of the fragment group in first cryptographic Hash divided by the index is calculated, obtains the first residual value；
The corresponding fragment group of first residual value is determined as the corresponding fragment group of the data.
Optionally, the ID according to the data determines that the data correspond in the corresponding fragment group of the data
Fragment, including：
Hash calculation is carried out to the ID of the data, obtains the second cryptographic Hash；
Second cryptographic Hash divided by the quantity remainder of the fragment in the corresponding fragment group of the data are calculated, obtains second
Residual value；
The corresponding fragment of second residual value is determined as the corresponding fragment of the data.
Optionally, this method further includes：
Routing value in searching request determines that described search asks corresponding fragment group；Wherein, described search is asked
To search for the request of data stored in the index；
Described search request is broadcast to whole fragments in the corresponding fragment group of described search request, it please to described search
Corresponding data is asked to scan for inquiring.
Optionally, this method further includes：
Utilize the fragment stored of each data and the correspondence of routing value in the pre-stored index, root
Determine that described search asks corresponding fragment according to the routing value in searching request；
Described search request is broadcast to described search and asks corresponding fragment, described search is asked corresponding data into
Row search inquiry.
In addition, the present invention also provides a kind of data storage device based on Elasticsearch indexes, including：
Pre-analysis module is filtered inquiry preanalysis for the data to acquisition, obtains the routing value in the data；
Wherein, field of the routing value for the default dimension in the data；
Determining module, for according to the routing value, determining the corresponding fragment group of the data in the index；
Memory module for the ID according to the data, determines the data pair in the corresponding fragment group of the data
The fragment answered, and store the data in the corresponding fragment of the data.
Optionally, the determining module, including：
First computational submodule for carrying out Hash calculation to the routing value, obtains the first cryptographic Hash；
First remainder submodule, the quantity for calculating the fragment group in first cryptographic Hash divided by the index take
It is remaining, obtain the first residual value；
First determination sub-module, for the corresponding fragment group of first residual value to be determined as the corresponding fragment of the data
Group.
Optionally, the memory module, including：
Second computational submodule for carrying out Hash calculation to the ID of the data, obtains the second cryptographic Hash；
Second remainder submodule, for calculating the fragment in second cryptographic Hash divided by the corresponding fragment group of the data
Quantity remainder, obtain the second residual value；
Second determination sub-module, for the corresponding fragment of second residual value to be determined as the corresponding fragment of the data.
Optionally, which further includes：
Determining module is searched for, for the routing value in searching request, determines that described search asks corresponding fragment group；
Wherein, described search request is searches for the request of data stored in the index；
Search inquiry module, for described search request to be broadcast to the whole in the corresponding fragment group of described search request
Fragment asks corresponding data to scan for inquiring described search.
A kind of date storage method based on Elasticsearch indexes provided by the present invention, including：To the number of acquisition
According to inquiry preanalysis is filtered, the routing value in data is obtained；Wherein, field of the routing value for the default dimension in data；
According to routing value, the corresponding fragment group of data is determined in the index；According to the ID of data, determined in the corresponding fragment group of data
The corresponding fragment of data, and store data in the corresponding fragment of data；
As it can be seen that the present invention by according to routing value, determining the corresponding fragment group of data in the index, it may be determined that storage number
According to fragment group, divide index using to the mode that fragment is grouped, change the storage and distribution mode of data, so as to
The consumption of performance and the expense of network when reducing data search, improve data search efficiency.In addition, the present invention also provides one kind
Based on the data storage device of Elasticsearch indexes, equally with above-mentioned advantageous effect.
Description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of invention, for those of ordinary skill in the art, without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
The stream of a kind of date storage method based on Elasticsearch indexes that Fig. 1 is provided by the embodiment of the present invention
Cheng Tu；
The knot of a kind of data storage device based on Elasticsearch indexes that Fig. 2 is provided by the embodiment of the present invention
Composition.
Specific embodiment
Purpose, technical scheme and advantage to make the embodiment of the present invention are clearer, below in conjunction with the embodiment of the present invention
In attached drawing, the technical solution in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is
Part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
All other embodiments obtained without making creative work shall fall within the protection scope of the present invention.
It please refers to Fig.1, Fig. 1 is deposited by a kind of data based on Elasticsearch indexes that the embodiment of the present invention provides
The flow chart of method for storing；This method can include：
Step 101：The data of acquisition are filtered with inquiry preanalysis, obtains the routing value in data；Wherein, routing value
Field for the default dimension in data.
It is understood that the purpose of this step can be to inquire preanalysis by the filter of data stored to needs, it will
The field of default dimension in the data filtered out is as routing value.It, can be by designing for presetting the specific choice of dimension
Personnel are voluntarily set according to practical scene and user demand, when can such as select user's query search data common dimension as
Default dimension, the present embodiment do not do this any restrictions.
Specifically, by user refund record data storage for, to user refund record data (Document) into
When line index stores, document number (receipt_no) this dimension in record of often being refunded according to business demand according to user
Data are scanned for inquiring, therefore, default dimension can be set as to document number, that is, filtering is obtained by this step
Document number in the data of user's refund record.
It should be noted that the detailed process in this step the data of acquisition are filtered with inquiry preanalysis, it can
Voluntarily to be set by designer, as long as the field of default dimension can be filtered out as routing value, the present embodiment does not do this
Any restrictions.
Step 102：According to routing value, the corresponding fragment group of data is determined in the index.
Wherein, the purpose of the present embodiment can be the segmentation index by being grouped to fragment in index, change data
Storage and distribution mode.Wherein, fragment in index is grouped, it can be to the data of search inquiry be needed to carry out default dimension
Analysis, the maximum data collection being likely to occur to presetting dimension are measured, so as to be indexed segmentation according to the maximum value.
Specifically, for the specific packet mode of fragment in index, that is, the specific partitioning scheme indexed, by designer
Member is voluntarily set according to usage scenario and user demand, can set the quantity difference of the fragment in index in each fragment group,
As according to needed in each fragment group for precalculating the data stored number be correspondingly arranged the quantity of fragment；It can also set
The quantity of fragment in index in each fragment group is identical, makes on the fragment that data are evenly distributed to as far as possible in fragment group, such as
Fragment group (Group) sum after segmentation is denoted as group_num, fragment (Shard) sum in index is denoted as shard_
Num, the mod for making shard_num%group_num is 0, ensures that the i.e. fragment quantity of size of each fragment group is identical, makes data
It is evenly distributed to as far as possible on each fragment of fragment group.For example, an index is provided with 32 fragments, then splitting scheme is only
It, when it is 1 to divide number, can be equivalently employed without being split, data are also distributed across 32 in [1,2,4,8,16,32] this section
In a fragment；When it is 32 to divide number, the size of each fragment group is 1, i.e., comprising 1 fragment, may make on a certain fragment
The overabundance of data of storage leads to data hot issue.As long as fragment in index can be grouped, for the specifically side of grouping
The quantity of fragment group and the quantity of the fragment in each fragment group in formula, that is, index, the present embodiment do not do any restrictions.
It is understood that the purpose of this step can be to match in index to store the data by the routing value of data
Fragment group.It for specific matching way, can voluntarily be set according to practical scene and user demand by designer, such as originally
Step can be to satisfy the need to carry out Hash calculation by value, obtain the first cryptographic Hash；Calculate the fragment in the first cryptographic Hash divided by index
The quantity remainder of group obtains the first residual value；The corresponding fragment group of first residual value is determined as the corresponding fragment group of data.Also
It is that Hash calculation is carried out to the routing value of data, modulus calculating then is carried out to the quantity of the fragment group in index, passes through mod
Value determines the corresponding fragment group of the data；Specifically, by taking the data storage of user's refund record as an example, to each Document
Receipt_no (document number) value of (i.e. the data of each refund record) carries out Hash calculating, and then group_num is taken
The data that mould, i.e. Hash (receipt_no) %groups_num, mod value are 0 can be routed to group 1, i.e. Shards [0,
1].As long as the fragment group that the data are stored in index can be matched according to the routing value of data, the present embodiment does not appoint this
What is limited.
Step 103：According to the ID of data, the corresponding fragment of data is determined in the corresponding fragment group of data, and in data
Data are stored in corresponding fragment.
It is understood that the purpose of this step can be to match storage in corresponding fragment group by the ID of data to be somebody's turn to do
The fragment of data, so as to store that data into corresponding fragment.It, can be by designer's root for specific matching way
It is factually voluntarily set with scene and user demand, the ID if this step can be logarithm evidence carries out Hash calculation, obtains second and breathes out
Uncommon value；The second cryptographic Hash divided by the quantity remainder of the fragment in the corresponding fragment group of data are calculated, obtains the second residual value；By second
The corresponding fragment of residual value is determined as the corresponding fragment of data.It is, the ID to data carries out Hash calculation, then to fragment group
In fragment quantity carry out modulus calculating, the corresponding fragment of the data is determined by mod values；Remember specifically, being refunded with user
For the data storage of record, Hash calculating is carried out to the ID of each Document (i.e. the data of each refund record), then
To quantity (group_size) modulus of fragment in fragment group, i.e. in Hash (receipt_no) %groups_num, group 1
The data that mod values are 0 can be routed to Shard 1.It is stored in corresponding fragment group as long as can be matched according to the ID of data
The fragment of the data, the present embodiment do not do this any restrictions.
In the present embodiment, the embodiment of the present invention, can by according to routing value, determining the corresponding fragment group of data in the index
To determine the fragment group of storage data, divide index using to the mode that fragment is grouped, change the storage and distribution of data
Mode, the consumption of performance and the expense of network during so as to reduce data search, improves data search efficiency.
Based on above-described embodiment, the date storage method based on Elasticsearch indexes that this implementation is provided may be used also
To include the step of corresponding data to storage scan for inquiry, the consumption of performance and network during reducing data search
Expense improves data search efficiency.
Specifically, the present embodiment can determine the corresponding fragment group of searching request according to the routing value in searching request；Its
In, searching request is the request of data stored in search index；Searching request is broadcast to the corresponding fragment group of searching request
In whole fragments, to searching request, corresponding data scan for inquiring.It is wide by Elasticsearch to avoid searching request
It is multicast in whole fragments in index, reduces the consumption of performance.
It should be noted that client takes the mode of single thread and multithreading to call respectively in testing
The query interface of Elasticsearch carries out 1000 inquiries, with Filter (filter) mistake of identical default dimension respectively
Filter, such as table 1 compared with the index segmentation routing algorithm for not utilizing the present embodiment indexes the data of segmentation routing algorithm with taking
It is shown：
The index segmentation routing algorithm effect displaying table of table 1
| Index divides Group numbers | Single thread (processing time s) | 2 threads | 4 threads |
| 2 | 2.86 | 4.31 | 7.15 |
| 4 | 2.07 | 3.04 | 4.85 |
| 0 (not dividing) | 4.02 | 6.25 | 11.18 |
The search efficiency that can be seen that after index segmentation from data comparison above is far above the efficiency of before processing.
Preferably, the fragment stored and routing value of each data in index can also be stored in a upper embodiment
Correspondence, the corresponding fragment of searching request is determined with the routing value directly in searching request, so as to by searching request
The corresponding fragment of searching request is broadcast to, corresponding data scan for inquiring to searching request, further promote search efficiency.
The present embodiment does not do this any restrictions.
In the present embodiment, the embodiment of the present invention is complete in the corresponding fragment group of searching request by the way that searching request is broadcast to
Part piece, to searching request, corresponding data scan for inquiring；It avoids searching request and rope is broadcast to by Elasticsearch
In the whole fragments drawn, the consumption of performance and the expense of network are reduced, improves data search efficiency.
It please refers to Fig.2, Fig. 2 is deposited by a kind of data based on Elasticsearch indexes that the embodiment of the present invention provides
The structure chart of storage device.The device can include：
Pre-analysis module 100 is filtered inquiry preanalysis for the data to acquisition, obtains the routing value in data；
Wherein, field of the routing value for the default dimension in data；
Determining module 200, for according to routing value, determining the corresponding fragment group of data in the index；
Memory module 300 for the ID according to data, determines the corresponding fragment of data in the corresponding fragment group of data,
And store data in the corresponding fragment of data.
Optionally, determining module 200 can include：
First computational submodule for carrying out Hash calculation to routing value, obtains the first cryptographic Hash；
First remainder submodule for calculating the quantity remainder of the fragment group in the first cryptographic Hash divided by index, obtains the
One residual value；
First determination sub-module, for the corresponding fragment group of the first residual value to be determined as the corresponding fragment group of data.
Optionally, memory module 300 can include：
Second computational submodule for carrying out Hash calculation to the ID of data, obtains the second cryptographic Hash；
Second remainder submodule, the quantity for calculating the fragment in the second cryptographic Hash divided by the corresponding fragment group of data take
It is remaining, obtain the second residual value；
Second determination sub-module, for the corresponding fragment of the second residual value to be determined as the corresponding fragment of data.
Optionally, which can also include：
Determining module is searched for, for the routing value in searching request, determines the corresponding fragment group of searching request；Its
In, searching request is the request of data stored in search index；
Search inquiry module is right for searching request to be broadcast to whole fragments in the corresponding fragment group of searching request
The corresponding data of searching request scan for inquiring.
In the present embodiment, the embodiment of the present invention, according to routing value, determines that data correspond in the index by determining module 200
Fragment group, it may be determined that store the fragment group of data, divide index using to the mode that fragment is grouped, change data
Storage and distribution mode, the consumption of performance and the expense of network during so as to reduce data search improve data search efficiency.
Each embodiment is described by the way of progressive in specification, the highlights of each of the examples are with other realities
Apply the difference of example, just to refer each other for identical similar portion between each embodiment.For device disclosed in embodiment
Speech, since it is corresponded to the methods disclosed in the examples, so description is fairly simple, related part is referring to method part illustration
.
Professional further appreciates that, with reference to each exemplary unit of the embodiments described herein description
And algorithm steps, can be realized with the combination of electronic hardware, computer software or the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is performed actually with hardware or software mode, specific application and design constraint depending on technical solution.Profession
Technical staff can realize described function to each specific application using distinct methods, but this realization should not
Think beyond the scope of this invention.
It can directly be held with reference to the step of method or algorithm that the embodiments described herein describes with hardware, processor
The combination of capable software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
The date storage method provided by the present invention based on Elasticsearch indexes and device have been carried out in detail above
It is thin to introduce.Specific case used herein is expounded the principle of the present invention and embodiment, and above example is said
The bright method and its core concept for being merely used to help understand the present invention.It should be pointed out that the ordinary skill for the art
For personnel, without departing from the principle of the present invention, can also to the present invention some improvement and modification can also be carried out, these improvement
It is also fallen within the protection scope of the claims of the present invention with modification.
Claims (10)
1. a kind of date storage method based on Elasticsearch indexes, which is characterized in that including：
Inquiry preanalysis is filtered to the data of acquisition, obtains the routing value in the data；Wherein, the routing value is institute
State the field of the default dimension in data；
According to the routing value, the corresponding fragment group of the data is determined in the index；
According to the ID of the data, the corresponding fragment of the data is determined in the corresponding fragment group of the data, and described
The data are stored in the corresponding fragment of data.
2. the date storage method according to claim 1 based on Elasticsearch indexes, which is characterized in that described
The quantity of fragment group in index is greater than or equal to 2, the quantity of the fragment in each fragment group in the index be all higher than or
It is equal to 2 and equal.
3. the date storage method according to claim 1 based on Elasticsearch indexes, which is characterized in that described
According to the routing value, the corresponding fragment group of the data is determined in the index, including：
Hash calculation is carried out to the routing value, obtains the first cryptographic Hash；
The quantity remainder of the fragment group in first cryptographic Hash divided by the index is calculated, obtains the first residual value；
The corresponding fragment group of first residual value is determined as the corresponding fragment group of the data.
4. the date storage method according to claim 3 based on Elasticsearch indexes, which is characterized in that described
According to the ID of the data, the corresponding fragment of the data is determined in the corresponding fragment group of the data, including：
Hash calculation is carried out to the ID of the data, obtains the second cryptographic Hash；
Second cryptographic Hash divided by the quantity remainder of the fragment in the corresponding fragment group of the data are calculated, obtains more than second
Value；
The corresponding fragment of second residual value is determined as the corresponding fragment of the data.
5. the date storage method according to any one of claims 1 to 4 based on Elasticsearch indexes, feature
It is, further includes：
Routing value in searching request determines that described search asks corresponding fragment group；Wherein, described search request is searches
The request of the data stored in Suo Suoshu indexes；
Described search request is broadcast to whole fragments in the corresponding fragment group of described search request, to described search request pair
The data answered scan for inquiring.
6. the date storage method according to any one of claims 1 to 4 based on Elasticsearch indexes, feature
It is, further includes：
Using the fragment stored of each data and the correspondence of routing value in the pre-stored index, according to searching
Routing value in rope request determines that described search asks corresponding fragment；
Described search request is broadcast to described search and asks corresponding fragment, corresponding data is asked to be searched described search
Rope is inquired.
7. a kind of data storage device based on Elasticsearch indexes, which is characterized in that including：
Pre-analysis module is filtered inquiry preanalysis for the data to acquisition, obtains the routing value in the data；Its
In, the routing value is the field of the default dimension in the data；
Determining module, for according to the routing value, determining the corresponding fragment group of the data in the index；
Memory module for the ID according to the data, determines that the data are corresponding in the corresponding fragment group of the data
Fragment, and store the data in the corresponding fragment of the data.
8. the data storage device according to claim 7 based on Elasticsearch indexes, which is characterized in that described
Determining module, including：
First computational submodule for carrying out Hash calculation to the routing value, obtains the first cryptographic Hash；
First remainder submodule for calculating the quantity remainder of the fragment group in first cryptographic Hash divided by the index, obtains
Take the first residual value；
First determination sub-module, for the corresponding fragment group of first residual value to be determined as the corresponding fragment group of the data.
9. the data storage device according to claim 8 based on Elasticsearch indexes, which is characterized in that described
Memory module, including：
Second computational submodule for carrying out Hash calculation to the ID of the data, obtains the second cryptographic Hash；
Second remainder submodule, for calculating the number of the fragment in second cryptographic Hash divided by the corresponding fragment group of the data
More than measurement, the second residual value is obtained；
Second determination sub-module, for the corresponding fragment of second residual value to be determined as the corresponding fragment of the data.
10. the data storage device according to any one of claims 7 to 9 based on Elasticsearch indexes, feature
It is, further includes：
Determining module is searched for, for the routing value in searching request, determines that described search asks corresponding fragment group；Its
In, described search request is searches for the request of data stored in the index；
Search inquiry module asks the whole in corresponding fragment group to be divided for described search request to be broadcast to described search
Piece asks corresponding data to scan for inquiring described search.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810063307.4A CN108197296B (en) | 2018-01-23 | 2018-01-23 | Data storage method based on Elasticissearch index |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810063307.4A CN108197296B (en) | 2018-01-23 | 2018-01-23 | Data storage method based on Elasticissearch index |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108197296A true CN108197296A (en) | 2018-06-22 |
| CN108197296B CN108197296B (en) | 2020-09-29 |
Family
ID=62590740
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810063307.4A Active CN108197296B (en) | 2018-01-23 | 2018-01-23 | Data storage method based on Elasticissearch index |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108197296B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109033248A (en) * | 2018-07-05 | 2018-12-18 | 华为技术有限公司 | The method and apparatus of storing data record, the method and apparatus for inquiring data record |
| CN109582758A (en) * | 2018-12-06 | 2019-04-05 | 重庆邮电大学 | A kind of Elasticsearch index fragment optimization method |
| CN109634917A (en) * | 2018-12-24 | 2019-04-16 | 成都四方伟业软件股份有限公司 | Method for reading data and device |
| CN109885642A (en) * | 2019-02-18 | 2019-06-14 | 国家计算机网络与信息安全管理中心 | Classification storage method and device towards full-text search |
| CN110569302A (en) * | 2019-08-16 | 2019-12-13 | 苏宁云计算有限公司 | method and device for physical isolation of distributed cluster based on lucene |
| CN111147332A (en) * | 2019-12-29 | 2020-05-12 | 北京浪潮数据技术有限公司 | Method, system and related device for determining uploading progress of cloud backup of storage system |
| CN112612865A (en) * | 2020-12-17 | 2021-04-06 | 杭州迪普科技股份有限公司 | Document storage method and device based on elastic search |
| DE102020117572A1 (en) | 2020-05-29 | 2021-12-02 | Shanghai Inhub Technology Co., Ltd. | Mobile radio communication device with two integrated subscriber identity modules for redundant communication |
| CN114020986A (en) * | 2022-01-05 | 2022-02-08 | 深圳思谋信息科技有限公司 | Content retrieval system |
| CN115577148A (en) * | 2022-12-08 | 2023-01-06 | 平安银行股份有限公司 | Batch data processing method, computer device and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008150638A3 (en) * | 2007-05-18 | 2009-03-19 | Microsoft Corp | Filtering of multi attribute data via on-demand indexing |
| CN103488687A (en) * | 2013-09-02 | 2014-01-01 | 用友软件股份有限公司 | Searching system and searching method of big data |
| CN104503708A (en) * | 2014-12-29 | 2015-04-08 | 成都致云科技有限公司 | Data hash storage method and device |
| CN104572785A (en) * | 2013-10-29 | 2015-04-29 | 阿里巴巴集团控股有限公司 | Method and device for establishing index in distributed form |
| CN104778252A (en) * | 2015-04-16 | 2015-07-15 | 天脉聚源(北京)传媒科技有限公司 | Index storage method and index storage device |
| CN106484877A (en) * | 2016-10-14 | 2017-03-08 | 东北大学 | A kind of document retrieval system based on HDFS |
| CN107515878A (en) * | 2016-06-16 | 2017-12-26 | 苏宁云商集团股份有限公司 | The management method and device of a kind of data directory |
-
2018
- 2018-01-23 CN CN201810063307.4A patent/CN108197296B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008150638A3 (en) * | 2007-05-18 | 2009-03-19 | Microsoft Corp | Filtering of multi attribute data via on-demand indexing |
| CN103488687A (en) * | 2013-09-02 | 2014-01-01 | 用友软件股份有限公司 | Searching system and searching method of big data |
| CN104572785A (en) * | 2013-10-29 | 2015-04-29 | 阿里巴巴集团控股有限公司 | Method and device for establishing index in distributed form |
| CN104503708A (en) * | 2014-12-29 | 2015-04-08 | 成都致云科技有限公司 | Data hash storage method and device |
| CN104778252A (en) * | 2015-04-16 | 2015-07-15 | 天脉聚源(北京)传媒科技有限公司 | Index storage method and index storage device |
| CN107515878A (en) * | 2016-06-16 | 2017-12-26 | 苏宁云商集团股份有限公司 | The management method and device of a kind of data directory |
| CN106484877A (en) * | 2016-10-14 | 2017-03-08 | 东北大学 | A kind of document retrieval system based on HDFS |
Non-Patent Citations (2)
| Title |
|---|
| MIQI1227: "Elasticsearch学习笔记（三）Elasticsearch集群分片的读写操作", 《HTTPS://WWW.WEBFALSE.COM/READ/205811/3605916.HTML》 * |
| 德普优化资讯: "Elasticsearch分片路由（Routing）配置", 《HTTPS://WWW.DEEPASO.COM/BLOG/162.HTML》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109033248B (en) * | 2018-07-05 | 2021-01-12 | 华为技术有限公司 | Method and device for storing data record and method and device for inquiring data record |
| CN109033248A (en) * | 2018-07-05 | 2018-12-18 | 华为技术有限公司 | The method and apparatus of storing data record, the method and apparatus for inquiring data record |
| CN109582758A (en) * | 2018-12-06 | 2019-04-05 | 重庆邮电大学 | A kind of Elasticsearch index fragment optimization method |
| CN109582758B (en) * | 2018-12-06 | 2023-04-07 | 重庆邮电大学 | Optimization method for Elasticissearch index shards |
| CN109634917A (en) * | 2018-12-24 | 2019-04-16 | 成都四方伟业软件股份有限公司 | Method for reading data and device |
| CN109885642A (en) * | 2019-02-18 | 2019-06-14 | 国家计算机网络与信息安全管理中心 | Classification storage method and device towards full-text search |
| CN109885642B (en) * | 2019-02-18 | 2021-11-02 | 国家计算机网络与信息安全管理中心 | Hierarchical storage method and device for full-text retrieval |
| CN110569302A (en) * | 2019-08-16 | 2019-12-13 | 苏宁云计算有限公司 | method and device for physical isolation of distributed cluster based on lucene |
| CN111147332A (en) * | 2019-12-29 | 2020-05-12 | 北京浪潮数据技术有限公司 | Method, system and related device for determining uploading progress of cloud backup of storage system |
| CN111147332B (en) * | 2019-12-29 | 2022-04-29 | 北京浪潮数据技术有限公司 | Method, system and related device for determining uploading progress of cloud backup of storage system |
| DE102020117572A1 (en) | 2020-05-29 | 2021-12-02 | Shanghai Inhub Technology Co., Ltd. | Mobile radio communication device with two integrated subscriber identity modules for redundant communication |
| DE102020117572B4 (en) | 2020-05-29 | 2021-12-23 | Shanghai Inhub Technology Co., Ltd. | Mobile radio communication device with two integrated subscriber identity modules for redundant communication |
| CN112612865A (en) * | 2020-12-17 | 2021-04-06 | 杭州迪普科技股份有限公司 | Document storage method and device based on elastic search |
| CN114020986A (en) * | 2022-01-05 | 2022-02-08 | 深圳思谋信息科技有限公司 | Content retrieval system |
| CN115577148A (en) * | 2022-12-08 | 2023-01-06 | 平安银行股份有限公司 | Batch data processing method, computer device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108197296B (en) | 2020-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108197296A (en) | Date storage method based on Elasticsearch indexes | |
| CN105229980B (en) | Utilize the method and apparatus of the exact-match lookup of variable keyword size | |
| US10198363B2 (en) | Reducing data I/O using in-memory data structures | |
| US6263364B1 (en) | Web crawler system using plurality of parallel priority level queues having distinct associated download priority levels for prioritizing document downloading and maintaining document freshness | |
| JP3485262B2 (en) | Method and means for classifying data packets | |
| US20160203179A1 (en) | Systems and methods for rapid data analysis | |
| EP3236365A1 (en) | Data query method and device | |
| CN104915717B (en) | Data processing method, Analysis of Knowledge Bases Reasoning method and relevant apparatus | |
| US7478083B2 (en) | Method and system for estimating cardinality in a database system | |
| CN108470364A (en) | A kind of curve-fitting method and device | |
| CN107783985B (en) | Distributed database query method, device and management system | |
| WO2017092583A1 (en) | Method and device for establishing index | |
| CN106815258A (en) | A kind of date storage method and coordinator node | |
| CN112131356B (en) | Message keyword matching method and device based on TCAM | |
| CN105550180B (en) | The method, apparatus and system of data processing | |
| US9400793B2 (en) | Model for generating custom file plans towards management of content as records | |
| CN105912696A (en) | DNS (Domain Name System) index creating method and query method based on logarithm merging | |
| CN105359142B (en) | Hash connecting method and device | |
| CN108182204A (en) | The processing method and processing device of data query based on house prosperity transaction multi-dimensional data | |
| CN110928900B (en) | Multi-table data query method, device, terminal and computer storage medium | |
| CN110784553A (en) | Message encapsulation method, device and domain name resolution system | |
| EP3203388B1 (en) | Database table index | |
| CN106940715B (en) | A kind of method and apparatus of the inquiry based on concordance list | |
| CN111905378B (en) | Data updating system, data updating method and server | |
| JP6980412B2 (en) | Systems and methods for efficient interval search using locality retention hashes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |