CN111428114B - Index creation method and device of elastic search engine - Google Patents
Index creation method and device of elastic search engine Download PDFInfo
- Publication number
- CN111428114B CN111428114B CN202010229745.0A CN202010229745A CN111428114B CN 111428114 B CN111428114 B CN 111428114B CN 202010229745 A CN202010229745 A CN 202010229745A CN 111428114 B CN111428114 B CN 111428114B
- Authority
- CN
- China
- Prior art keywords
- index
- elastic search
- resource group
- request
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000002955 isolation Methods 0.000 claims abstract description 10
- 230000015654 memory Effects 0.000 claims description 22
- 238000012508 change request Methods 0.000 claims description 19
- 230000008859 change Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 13
- 238000012545 processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 3
- 229920001971 elastomer Polymers 0.000 description 3
- 239000000806 elastomer Substances 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013138 pruning Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an index creation method and device of an elastic search engine, wherein the method comprises the following steps: obtaining an index creation request, wherein the index creation request comprises: resource group information; determining the elastic search node information corresponding to the resource group information; generating index configuration according to the elastic search node information; and sending the index configuration to an elastic search node in the elastic search node information so that the elastic search node generates an index corresponding to the index creation request according to the index configuration. The invention realizes the resource isolation of the elastic search, and solves the problem that when one tenant uses a large amount of node resources, the cluster can not provide the computing capability or the service capability to other tenants to be reduced.
Description
Technical Field
The invention relates to an elastic search engine, in particular to an index creation method and device of the elastic search engine.
Background
The elastiscearch is an excellent Lucene-based distributed search engine. In the current setting of large data technology used by enterprises, an elastic search cluster is used in many cases to provide search services. An elastic search cluster stores data in one or more indexes, which are collections of documents with similar characteristics. Currently, the elastomer search does not have the property of resource isolation, and the index in the elastomer search cluster can use all elastomer search nodes in the cluster. This results in resource contention in the case of multi-tenant use, which may result in clusters that fail to provide computing power or service power to other tenants when one tenant uses node resources in large amounts.
Disclosure of Invention
The invention provides an index creation method and device of an elastic search engine in order to solve at least one technical problem in the background art.
In order to achieve the above object, according to one aspect of the present invention, there is provided an index creation method of an elastic search engine, the method comprising:
obtaining an index creation request, wherein the index creation request comprises: resource group information;
determining the elastic search node information corresponding to the resource group information;
generating index configuration according to the elastic search node information;
and sending the index configuration to an elastic search node in the elastic search node information so that the elastic search node generates an index corresponding to the index creation request according to the index configuration.
Optionally, the index creation request further includes: a first index name of the index to be created;
the index creation method of the elastic search engine further comprises the following steps:
and generating a second index name of the index to be created according to the resource group information and the first index name of the index to be created.
Optionally, the sending the index configuration to an elastic search node in the elastic search node information, so that the elastic search node generates an index corresponding to the index creation request according to the index configuration, specifically:
and sending the index configuration and the second index name to an elastic search node in the elastic search node information so that the elastic search node generates the index to be created according to the index configuration and the second index name.
Optionally, the index creating method of the elastic search engine further includes:
obtaining a first operation request for performing an adding, deleting and checking operation on data in a target index, wherein the first operation request comprises: resource group information and a first index name of the target index;
determining the elastic search node information corresponding to the resource group information in the first operation request;
determining a second index name of the target index according to the resource group information in the first operation request and the first index name of the target index;
generating a second operation request according to the second index name and the first operation request, and sending the second operation request to an elastic search node in the elastic search node information corresponding to the resource group information in the first operation request.
Optionally, the index creating method of the elastic search engine further includes:
obtaining a node change request of a target resource group, wherein the node change request is as follows: node change information;
determining all indexes corresponding to the target resource group;
and updating the index configuration of all indexes corresponding to the target resource group according to the node change information.
In order to achieve the above object, according to another aspect of the present invention, there is provided an index creation apparatus of an elastic search engine, the apparatus comprising:
an index creation request acquisition unit configured to acquire an index creation request, where the index creation request includes: resource group information;
a node determining unit, configured to determine elastic search node information corresponding to the resource group information;
an index configuration generating unit, configured to generate an index configuration according to the elastic search node information;
and an index creation information sending unit, configured to send the index configuration to an elastic search node in the elastic search node information, so that the elastic search node generates an index corresponding to the index creation request according to the index configuration.
Optionally, the index creation request further includes: a first index name of the index to be created;
the index creation device of the elastic search engine further comprises:
and the index name generation unit is used for generating a second index name of the index to be created according to the resource group information and the first index name of the index to be created.
Optionally, the index creation information sending unit is specifically configured to send the index configuration and the second index name to an elastic search node in the elastic search node information, so that the elastic search node generates the index to be created according to the index configuration and the second index name.
Optionally, the index creating device of the elastic search engine further includes:
a data adding/deleting request obtaining unit, configured to obtain a first operation request for adding/deleting data in a target index, where the first operation request includes: resource group information and a first index name of the target index;
a node information determining unit, configured to determine elastic search node information corresponding to the resource group information in the first operation request;
an index name determining unit, configured to determine a second index name of the target index according to the resource group information in the first operation request and the first index name of the target index;
an operation request sending unit, configured to generate a second operation request according to the second index name and the first operation request, and send the second operation request to an elastic search node in elastic search node information corresponding to resource group information in the first operation request.
Optionally, the index creating device of the elastic search engine further includes:
a node change request obtaining unit, configured to obtain a node change request of a target resource group, where the node change request is: node change information;
an index query unit, configured to determine all indexes corresponding to the target resource group;
and the index configuration updating unit is used for updating the index configuration of all indexes corresponding to the target resource group according to the node change information.
To achieve the above object, according to another aspect of the present invention, there is also provided a computer device including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the index creating method of the above-mentioned elastic search engine when executing the computer program.
In order to achieve the above object, according to another aspect of the present invention, there is also provided a computer-readable storage medium storing a computer program which, when executed in a computer processor, implements the steps in the index creation method of the above-mentioned elastic search engine.
The beneficial effects of the invention are as follows: according to the embodiment of the invention, by setting the resource group, each tenant corresponds to one resource group, configuring a plurality of elastic search nodes corresponding to each resource group, writing the elastic search nodes corresponding to the resource group into the index configuration when generating the index corresponding to the resource group, and limiting the nodes which can be used by the generated index, the resource isolation of the elastic search is realized, and the problem that when one tenant uses a large amount of node resources, the cluster can not provide the computing capability or the service capability to other tenants is solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:
FIG. 1 is a flowchart of a first embodiment index creation method of the present invention;
FIG. 2 is a flow chart of a second embodiment index creation method of the present invention;
FIG. 3 is a flowchart of a first embodiment of the present invention for performing a pruned operation on data in an index;
FIG. 4 is a flowchart of a change to a node corresponding to a resource group according to an embodiment of the present invention;
FIG. 5 is a flowchart of an index creation method according to a third embodiment of the present invention;
FIG. 6 is a flow chart of a second embodiment of the invention for performing a pruned operation on data in an index;
FIG. 7 is a schematic view of a usage scenario of an index creation method according to an embodiment of the present invention;
FIG. 8 is a block diagram showing the structure of an index creating apparatus according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of a computer device according to an embodiment of the invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
It is noted that the terms "comprises" and "comprising," and any variations thereof, in the description and claims of the present invention and in the foregoing figures, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that, without conflict, the embodiments of the present invention and features of the embodiments may be combined with each other. The invention will be described in detail below with reference to the drawings in connection with embodiments.
The invention provides an index creation method of an elastic search engine, which realizes resource isolation of the elastic search. Fig. 7 is a schematic view of a usage scenario of the index creating method according to an embodiment of the present invention, and as shown in fig. 7, the present invention implements resource isolation by adopting a manner of customizing a client and adding a service agent module. By the method, the operation and maintenance cost in the use process of the enterprise can be reduced.
When the invention uses the elastic search, the resources to be isolated define different 'Resource groups', one 'Resource Group' corresponds to a plurality of elastic search nodes, and each 'Resource Group' can only use the corresponding elastic search node in the elastic search cluster. Optionally, one "resource group" corresponds to at least 3 elastic search nodes.
As shown in fig. 7, the usage scenario of the index creating method according to the embodiment of the present invention includes: client, service agent module, metadata storage module, and elastic search cluster.
And the client is used by the tenant to connect with the service proxy module. When the client side performs instantiation, the resource group name can be input, and the following interfaces can be provided:
the service agent module is used for providing an accessed server to the outside, receiving and processing a request from a client, and interacting with the metadata storage module to acquire the metadata stored with the relation among the resource group, the node, the physical machine or the virtual machine, and performing data operation with the elastic search cluster.
The metadata storage module may be a database, and a metadata table for recording the relation among the resource group, the node, the physical machine or the virtual machine in the database includes 3 fields of node name, IP address and resource group name, and the table construction statement may be as follows:
fig. 1 is a flowchart of an index creating method of an elastic search engine according to an alternative embodiment of the present invention, and as shown in fig. 1, the index creating method of an elastic search engine according to the present embodiment includes steps S101 to S104.
Step S101, obtaining an index creation request, where the index creation request includes: resource group information.
In the embodiment of the invention, the service agent module receives an index creation request sent by a user through a client. And each user corresponds to one resource group, and the index creation request comprises resource group information corresponding to the user. In an alternative embodiment of the present invention, the resource group information may be a resource group name or a resource group number.
Step S102, determining the elastic search node information corresponding to the resource group information.
In the embodiment of the invention, each resource group corresponds to a plurality of preset elastic search nodes. The correspondence between the resource group, the elastic search node, the physical machine, or the virtual machine is stored in the metadata storage module. The service agent module can query the elastic search node information corresponding to the resource group information by accessing the metadata storage module.
And step S103, generating index configuration according to the elastic search node information.
In this step, the service proxy module writes the elastic search node information corresponding to the resource group into the index configuration of the index to be generated, and the index configuration limits the elastic search nodes that can be used by the index to be generated, thereby realizing resource isolation.
And step S104, the index configuration is sent to an elastic search node in the elastic search node information, so that the elastic search node generates an index corresponding to the index creation request according to the index configuration.
In this step, the service agent module transmits the index configuration to any one of the above-described elastic search node information in the elastic search cluster, which creates a corresponding index according to the index configuration.
Therefore, according to the embodiment of the invention, by setting the resource group, each tenant corresponds to one resource group, configuring a plurality of elastic search nodes corresponding to each resource group, writing the elastic search nodes corresponding to the resource group into the index configuration when the index corresponding to the resource group is generated, and limiting the nodes which can be used by the generated index, the resource isolation of the elastic search is realized, and the problem that the cluster can not provide the computing capacity or the service capacity to other tenants to be reduced when one tenant uses a large amount of node resources at present is solved.
Fig. 2 is a flowchart of an index creating method of an elastic search engine according to another alternative embodiment of the present invention, and as shown in fig. 2, the index creating method of the elastic search engine of the present embodiment includes steps S201 to S205.
Step S201, obtaining an index creation request, where the index creation request includes: resource group information and a first index name of an index to be created.
In an alternative embodiment of the present invention, the first index name is an index name of an index to be created set by a user.
Step S202, determining the elastic search node information corresponding to the resource group information.
Step S203, generating index configuration according to the elastic search node information.
Step S204, generating a second index name of the index to be created according to the resource group information and the first index name of the index to be created.
In an alternative embodiment of the present invention, the step may splice the resource group information with the first index name to generate the second index name. Specifically, the resource group information may be used as a prefix of the first index name to generate the second index name. In the embodiment of the invention, the second index name is the index name actually in the elastic search cluster.
In the embodiment of the invention, one index comprises two index names, namely an index name set by a user and an index name actually in an elastic search cluster. When the client interacts with the service proxy module, the index name set by the user is used in the data, and when the service proxy module interacts with the elastic search cluster, the index name actually in the elastic search cluster is used in the data, and the corresponding relation between the index name set by the user and the index name actually in the elastic search cluster can be stored in the service proxy module. By adopting the method, the index with the same name can be effectively prevented from being created in the elastic search cluster by different users, and the user can conveniently manage the index created by the user.
Step S205, transmitting the index configuration and the second index name to an elastic search node in the elastic search node information, so that the elastic search node generates the index to be created according to the index configuration and the second index name.
In this step, the service proxy module transmits the index configuration and the second index name to any one of the above-mentioned elastic search node information in the elastic search cluster, and the elastic search node creates a corresponding index according to the index configuration and the second index name.
Fig. 3 is a flowchart of an operation of adding, deleting and checking data in an index according to a first embodiment of the present invention, and as shown in fig. 3, in an embodiment of the present invention, the flow of the operation of adding, deleting and checking data in an index includes steps S301 to S304.
Step S301, obtaining a first operation request for performing an adding, deleting and checking operation on data in a target index, where the first operation request includes: resource group information and a first index name of the target index.
In the embodiment of the invention, the service agent module receives the first operation request sent by the user through the client. In an alternative embodiment of the present invention, the first operation request may be a data query request for the target index, a data write request for the target index, a data delete request for the target index, and a data modification request for the target index. The first operation request also comprises data processing information.
Step S302, determining elastic search node information corresponding to the resource group information in the first operation request.
Step S303, determining a second index name of the target index according to the resource group information in the first operation request and the first index name of the target index.
In the embodiment of the present invention, in this step, the resource group information may be used as a prefix of the first index name to obtain the second index name of the target index. In other optional embodiments of the present invention, a corresponding relationship between each first index name and each second index name in each resource group may be stored in the service proxy module, and this step may determine, according to the corresponding relationship, a second index name corresponding to a certain first index name in a certain resource group.
Step S304, generating a second operation request according to the second index name and the first operation request, and sending the second operation request to an elastic search node in the elastic search node information corresponding to the resource group information in the first operation request.
In an alternative embodiment of the present invention, the step may replace the first index name in the first operation request with the second index name, to obtain the second operation request. In this step, the service proxy module sends the second operation request to any one of the above-mentioned elastic search node information in the elastic search cluster, and then the elastic search node finds the target index according to the second index name in the second operation request, and further executes the data processing operation corresponding to the second operation request.
Fig. 4 is a flowchart of changing a node corresponding to a resource group according to an embodiment of the present invention, and as shown in fig. 4, in an alternative embodiment of the present invention, a flow of changing a node corresponding to a resource group includes steps S401 to S403.
Step S401, obtaining a node change request of a target resource group, where the node change request is: node change information.
In the embodiment of the invention, the service agent module receives a node change request sent by a user through a client. In an alternative embodiment of the present invention, the node change request may be a request for adding an elastic search node corresponding to the target resource group, a request for deleting a certain or some of the elastic search nodes corresponding to the target resource group, or a request for replacing a certain or some of the elastic search nodes corresponding to the target resource group.
Step S402, determining all indexes corresponding to the target resource group.
In the embodiment of the invention, the service agent module sends a query request to the elastic search cluster to query all indexes corresponding to the target resource group. In an alternative embodiment of the present invention, since the name of the index (the second index name) in the elastic search cluster includes the prefix of the resource group information, the elastic search cluster may find all indexes corresponding to the target resource group by determining whether the prefix of the index name is identical to the target resource group.
Step S403, updating the index configuration of all indexes corresponding to the target resource group according to the node change information.
In the embodiment of the invention, after the elastic search cluster searches all indexes corresponding to the target resource group, the index configuration of each searched index is updated according to the node change information, so that the node change of the target resource group is completed.
In the embodiment of the invention, after the change of the node of the target resource group is completed, the metadata storage module updates the corresponding relation between the target resource group and the elastic search node.
Fig. 5 is a flowchart of an index creating method according to a third embodiment of the present invention, and in an alternative embodiment of the present invention, the flowchart of the index creating method according to the present invention may specifically include steps S501 to S509.
In step S501, the client sends a new index request to the service proxy module.
In step S502, after receiving the request, the service proxy module parses the resource group information from the request.
Step S503, according to step S502, if the service agent module obtains the resource group information, step S504 is executed; if the resource group information is not acquired, step S509 is performed.
Step S504, obtaining the node information of the resource group from the metadata storage module.
In step S505, the service agent module generates an index configuration, where the node used for restricting the index in the configuration is the node obtained in step S504.
In step S506, the service proxy module adds the resource group prefix to the index name to be newly created, and generates the index name in the actual cluster.
In step S507, the service proxy module sends a new index request to any one of the elastic search nodes corresponding to the resource group in the elastic search cluster, where the new index request includes the index configuration and an index name in the actual cluster.
In step S508, the service agent module packages the elastic search cluster return information, and returns the information to the client as success or failure information of the new index.
In step S509, the service proxy module returns a client error message informing that the request does not include the resource group information.
Fig. 6 is a flowchart of the operation of adding, deleting and checking data in the index according to the second embodiment of the present invention, as shown in fig. 6, in an alternative embodiment of the present invention, the flow of adding, deleting and checking data in the index specifically includes steps S601 to S608.
In step S601, the client sends a data adding/deleting request, where the data adding/deleting request includes an index name of an index where the data is located.
In step S602, after receiving the request, the service proxy module parses the resource group information from the request.
Step S603, according to step S602, if the service proxy module obtains the resource group information, step S604 is executed; if no resource group information is acquired, step S608 is performed.
In step S604, the service proxy module obtains the node information of the resource group from the metadata storage module.
Step S605, the service agent module adds a resource group prefix to the index name to obtain the index name in the cluster;
in step S606, the service agent module sends a data pruning query to any one of the elastic search nodes corresponding to the resource group in the elastic search cluster, where the data pruning query includes an index name in the cluster.
In step S607, the service agent module obtains the information returned by the elastic search cluster in step S606, and modifies the index name to remove the prefix of the resource group, and returns the information to the client as the return information of the data adding, deleting and checking operation.
In step S608, the service proxy module returns information of the client HTTP status code 404, which indicates the resource group without operation.
In an alternative embodiment of the present invention, the capacity expansion operation flow for the elastic search node corresponding to a certain resource group is as follows:
and step 1, the client sends a Rest request to the service proxy module, wherein the request contains the information of the contrast relation between the new node and the resource group.
And step 2, after receiving the request, the service agent module inserts the comparison relation between the new node and the resource group into the metadata storage module.
And 3, the service agent module sends a request to the elastic search cluster to acquire all indexes in the resource group.
And 4, the service agent module sends a request to the elastic search cluster to update the node range which is acquired in the step 3 and is allowed to be used for the index.
And step 5, the service agent module returns a success or failure message of the client.
From the above embodiments, it can be seen that the present invention achieves at least the following advantages: according to the method and the device, the resource groups are set, each tenant corresponds to one resource group, and the plurality of elastic search nodes are configured for each resource group, when the index corresponding to the resource group is generated, the elastic search nodes corresponding to the resource group are written into the index configuration, and the nodes which can be used by the generated index are limited, so that the resource isolation of the elastic search is realized, the operation and maintenance cost of a plurality of sets of clusters can be effectively reduced, and no extra burden is generated on a user.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowcharts, in some cases the steps illustrated or described may be performed in an order other than that illustrated herein.
Based on the same inventive concept, the embodiment of the present invention also provides an index creation device of an elastic search engine, which can be used to implement the index creation method of the elastic search engine described in the above embodiment, as described in the following embodiments. Since the principle of the index creating apparatus of the elastic search engine to solve the problem is similar to that of the index creating method of the elastic search engine, the embodiment of the index creating apparatus of the elastic search engine may refer to the embodiment of the index creating method of the elastic search engine, and the repetition is not repeated. As used below, the term "unit" or "module" may be a combination of software and/or hardware that implements the intended function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
Fig. 8 is a block diagram of an index creating apparatus of an elastic search engine according to an embodiment of the present invention, as shown in fig. 8, the index creating apparatus of an elastic search engine according to an embodiment of the present invention includes: an index creation request acquisition unit 1, a node determination unit 2, an index configuration generation unit 3, and an index creation information transmission unit 4.
An index creation request acquiring unit 1 configured to acquire an index creation request, where the index creation request includes: resource group information.
And a node determining unit 2, configured to determine elastic search node information corresponding to the resource group information.
An index configuration generating unit 3, configured to generate an index configuration according to the elastic search node information.
An index creation information transmitting unit 4, configured to transmit the index configuration to an elastic search node in the elastic search node information, so that the elastic search node generates an index corresponding to the index creation request according to the index configuration.
In an alternative embodiment of the present invention, the index creation request further includes: the first index name of the index is to be created. The index creation device of the elastic search engine further comprises: and the index name generation unit is used for generating a second index name of the index to be created according to the resource group information and the first index name of the index to be created.
In an optional embodiment of the present invention, the index creation information sending unit 4 is specifically configured to send the index configuration and the second index name to an elastic search node in the elastic search node information, so that the elastic search node generates the index to be created according to the index configuration and the second index name.
In an alternative embodiment of the present invention, the index creating apparatus of the elastic search engine of the present invention further includes:
a data adding/deleting request obtaining unit, configured to obtain a first operation request for adding/deleting data in a target index, where the first operation request includes: resource group information and a first index name of the target index;
a node information determining unit, configured to determine elastic search node information corresponding to the resource group information in the first operation request;
an index name determining unit, configured to determine a second index name of the target index according to the resource group information in the first operation request and the first index name of the target index;
an operation request sending unit, configured to generate a second operation request according to the second index name and the first operation request, and send the second operation request to an elastic search node in elastic search node information corresponding to resource group information in the first operation request.
In an alternative embodiment of the present invention, the index creating apparatus of the elastic search engine of the present invention further includes:
a node change request obtaining unit, configured to obtain a node change request of a target resource group, where the node change request is: node change information;
an index query unit, configured to determine all indexes corresponding to the target resource group;
and the index configuration updating unit is used for updating the index configuration of all indexes corresponding to the target resource group according to the node change information.
To achieve the above object, according to another aspect of the present application, there is also provided a computer apparatus. As shown in fig. 9, the computer device includes a memory, a processor, a communication interface, and a communication bus, where a computer program executable on the processor is stored on the memory, and when the processor executes the computer program, the steps in the method of the above embodiment are implemented.
The processor may be a central processing unit (Central Processing Unit, CPU). The processor may also be any other general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof.
The memory is used as a non-transitory computer readable storage medium for storing non-transitory software programs, non-transitory computer executable programs, and units, such as corresponding program units in the above-described method embodiments of the invention. The processor executes the various functional applications of the processor and the processing of the composition data by running non-transitory software programs, instructions and modules stored in the memory, i.e., implementing the methods of the method embodiments described above.
The memory may include a memory program area and a memory data area, wherein the memory program area may store an operating system, at least one application program required for a function; the storage data area may store data created by the processor, etc. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory may optionally include memory located remotely from the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more units are stored in the memory, which when executed by the processor, performs the method in the above embodiments.
The details of the computer device may be correspondingly understood by referring to the corresponding relevant descriptions and effects in the above embodiments, and will not be repeated here.
To achieve the above object, according to another aspect of the present application, there is also provided a computer-readable storage medium storing a computer program which, when executed in a computer processor, implements the steps in the index creation method of the above-mentioned elastic search engine. It will be appreciated by those skilled in the art that implementing all or part of the above-described embodiment method may be implemented by a computer program to instruct related hardware, where the program may be stored in a computer readable storage medium, and the program may include the above-described embodiment method when executed. The storage medium may be a magnetic Disk, an optical disc, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
It will be apparent to those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, or they may alternatively be implemented in program code executable by computing devices, such that they may be stored in a memory device for execution by the computing devices, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. An index creation method of an elastic search engine, comprising:
obtaining an index creation request sent by a tenant, wherein the index creation request comprises: resource group information, wherein each tenant corresponds to one resource group;
determining the elastic search node information corresponding to the resource group information;
generating an index configuration according to the elastic search node information, wherein the index configuration is used for limiting the elastic search nodes used by the index, so that resource isolation is realized;
transmitting the index configuration to an elastic search node in the elastic search node information so that the elastic search node generates an index corresponding to the index creation request according to the index configuration;
obtaining a node change request of a target resource group, wherein the node change request comprises: node change information; the node change request is a request for newly adding an elastic search node corresponding to the target resource group, a request for deleting one or more elastic search nodes corresponding to the target resource group, and a request for replacing one or more elastic search nodes corresponding to the target resource group;
determining all indexes corresponding to the target resource group, wherein all indexes corresponding to the target resource group are searched by judging whether the prefix of an index name is the same as that of the target resource group, and the names of the indexes in the elastic search cluster comprise resource group information prefixes;
and updating the index configuration of all indexes corresponding to the target resource group according to the node change information, thereby completing the node change of the target resource group.
2. The index creation method of an elastic search engine according to claim 1, wherein the index creation request further includes: a first index name of the index to be created;
the index creation method of the elastic search engine further comprises the following steps:
and generating a second index name of the index to be created according to the resource group information and the first index name of the index to be created.
3. The method for creating the index of the elastic search engine according to claim 2, wherein the sending the index configuration to the elastic search node in the elastic search node information, so that the elastic search node generates the index corresponding to the index creation request according to the index configuration, specifically is:
and sending the index configuration and the second index name to an elastic search node in the elastic search node information so that the elastic search node generates the index to be created according to the index configuration and the second index name.
4. The index creation method of an elastic search engine according to claim 3, further comprising:
obtaining a first operation request for performing an adding, deleting and checking operation on data in a target index, wherein the first operation request comprises: resource group information and a first index name of the target index;
determining the elastic search node information corresponding to the resource group information in the first operation request;
determining a second index name of the target index according to the resource group information in the first operation request and the first index name of the target index;
generating a second operation request according to the second index name and the first operation request, and sending the second operation request to an elastic search node in the elastic search node information corresponding to the resource group information in the first operation request.
5. An index creation device of an elastic search engine, comprising:
an index creation request obtaining unit, configured to obtain an index creation request sent by a tenant, where the index creation request includes: resource group information, wherein each tenant corresponds to one resource group;
a node determining unit, configured to determine elastic search node information corresponding to the resource group information;
an index configuration generating unit, configured to generate an index configuration according to the elastic search node information, where the index configuration is used to limit the elastic search nodes used by the index, so as to implement resource isolation;
an index creation information transmitting unit, configured to transmit the index configuration to an elastic search node in the elastic search node information, so that the elastic search node generates an index corresponding to the index creation request according to the index configuration;
a node change request obtaining unit, configured to obtain a node change request of a target resource group, where the node change request includes: node change information; the node change request is a request for newly adding an elastic search node corresponding to the target resource group, a request for deleting one or more elastic search nodes corresponding to the target resource group, and a request for replacing one or more elastic search nodes corresponding to the target resource group;
an index query unit, configured to determine all indexes corresponding to the target resource group, where by determining whether the prefix of the index name is the same as that of the target resource group, all indexes corresponding to the target resource group are searched, and the names of the indexes in the elastic search cluster include the resource group information prefix;
and the index configuration updating unit is used for updating the index configuration of all indexes corresponding to the target resource group according to the node change information, so that the node change of the target resource group is completed.
6. The index creation device of an elastic search engine according to claim 5, wherein the index creation request further includes: a first index name of the index to be created;
the index creation device of the elastic search engine further comprises:
and the index name generation unit is used for generating a second index name of the index to be created according to the resource group information and the first index name of the index to be created.
7. The index creation device of an elastic search engine according to claim 6, wherein the index creation information sending unit is specifically configured to send the index configuration and the second index name to an elastic search node in the elastic search node information, so that the elastic search node generates the index to be created according to the index configuration and the second index name.
8. The index creation device of an elastic search engine according to claim 7, further comprising:
a data adding/deleting request obtaining unit, configured to obtain a first operation request for adding/deleting data in a target index, where the first operation request includes: resource group information and a first index name of the target index;
a node information determining unit, configured to determine elastic search node information corresponding to the resource group information in the first operation request;
an index name determining unit, configured to determine a second index name of the target index according to the resource group information in the first operation request and the first index name of the target index;
an operation request sending unit, configured to generate a second operation request according to the second index name and the first operation request, and send the second operation request to an elastic search node in elastic search node information corresponding to resource group information in the first operation request.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 4 when executing the computer program.
10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed in a computer processor implements the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010229745.0A CN111428114B (en) | 2020-03-27 | 2020-03-27 | Index creation method and device of elastic search engine |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010229745.0A CN111428114B (en) | 2020-03-27 | 2020-03-27 | Index creation method and device of elastic search engine |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111428114A CN111428114A (en) | 2020-07-17 |
| CN111428114B true CN111428114B (en) | 2024-04-09 |
Family
ID=71548983
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010229745.0A Active CN111428114B (en) | 2020-03-27 | 2020-03-27 | Index creation method and device of elastic search engine |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111428114B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113568749B (en) * | 2021-07-28 | 2023-09-05 | 新华智云科技有限公司 | Method for distributing shards based on elastic search cluster |
| CN114564485A (en) * | 2022-04-28 | 2022-05-31 | 深圳竹云科技股份有限公司 | User data processing method based on Elastic Search |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109063077A (en) * | 2018-07-24 | 2018-12-21 | 新华三大数据技术有限公司 | A kind of data access method and device based on elasticity search |
| CN109144994A (en) * | 2017-06-19 | 2019-01-04 | 华为技术有限公司 | Index updating method, system and relevant apparatus |
| CN110019200A (en) * | 2017-09-30 | 2019-07-16 | 阿里巴巴集团控股有限公司 | A kind of foundation of index, application method and device |
| CN110569302A (en) * | 2019-08-16 | 2019-12-13 | 苏宁云计算有限公司 | method and device for physical isolation of distributed cluster based on lucene |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9270546B2 (en) * | 2014-03-05 | 2016-02-23 | Software Ag | Systems and/or methods for on-demand repository bootstrapping at runtime in a scalable, distributed multi-tenant environment |
-
2020
- 2020-03-27 CN CN202010229745.0A patent/CN111428114B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109144994A (en) * | 2017-06-19 | 2019-01-04 | 华为技术有限公司 | Index updating method, system and relevant apparatus |
| CN110019200A (en) * | 2017-09-30 | 2019-07-16 | 阿里巴巴集团控股有限公司 | A kind of foundation of index, application method and device |
| CN109063077A (en) * | 2018-07-24 | 2018-12-21 | 新华三大数据技术有限公司 | A kind of data access method and device based on elasticity search |
| CN110569302A (en) * | 2019-08-16 | 2019-12-13 | 苏宁云计算有限公司 | method and device for physical isolation of distributed cluster based on lucene |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111428114A (en) | 2020-07-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10715485B2 (en) | Managing dynamic IP address assignments | |
| US10467241B2 (en) | Dynamically provisioning instances of a single-tenant application for multi-tenant use | |
| CN102419770B (en) | File sharing system, method for realizing file sharing, and file index service equipment | |
| CN103067433B (en) | A kind of data migration method of distributed memory system, equipment and system | |
| CN106936899B (en) | Configuration method of distributed statistical analysis system and distributed statistical analysis system | |
| US11734248B2 (en) | Metadata routing in a distributed system | |
| CN105138592A (en) | Distributed framework-based log data storing and retrieving method | |
| US8706856B2 (en) | Service directory | |
| CN111428114B (en) | Index creation method and device of elastic search engine | |
| CN105045762A (en) | Management method and apparatus for configuration file | |
| CN110569302A (en) | method and device for physical isolation of distributed cluster based on lucene | |
| US11461053B2 (en) | Data storage system with separate interfaces for bulk data ingestion and data access | |
| US11108854B2 (en) | Peer-to-peer network for internet of things resource allocation operation | |
| US10261949B2 (en) | Packed row representation for efficient network serialization with direct column indexing in a network switch | |
| US20180293317A1 (en) | Prefix matching using distributed tables for storage services compatibility | |
| CN110597808B (en) | Distributed database table connection method, device, system, server and medium | |
| CN110798358B (en) | Distributed service identification method and device, computer readable medium and electronic equipment | |
| CN114466031B (en) | CDN system node configuration method, device, equipment and storage medium | |
| US11188419B1 (en) | Namespace indices in dispersed storage networks | |
| US11500700B2 (en) | Leasing prioritized items in namespace indices | |
| CN117390078B (en) | Data processing method, device, storage medium and computer equipment | |
| CN117075823B (en) | Object searching method, system, electronic device and storage medium | |
| US11194660B1 (en) | Listing and protocol for namespace index | |
| CN115878046A (en) | Data processing method, system, device, storage medium and electronic equipment | |
| Nuaimi et al. | A smart storage optimisation technique on the cloud |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |