CN109840266A - Storage system building method and device - Google Patents
Storage system building method and device Download PDFInfo
- Publication number
- CN109840266A CN109840266A CN201910072199.1A CN201910072199A CN109840266A CN 109840266 A CN109840266 A CN 109840266A CN 201910072199 A CN201910072199 A CN 201910072199A CN 109840266 A CN109840266 A CN 109840266A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- data center
- content
- interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention proposes a kind of storage system building method and device, and wherein method includes: to obtain the pending multiple data centers built；At least two nodes of ES cluster are set in each data center, are provided at least one main fragment in node, for storing data the data at center；Data include: alarm configuration data and alarm data；According to the node being arranged in each data center, the ES cluster in multiple data centers is built in generation, so as to the processing such as be stored, inquired to the alarm data of multiple data centers using ES cluster, and even if some data center disconnects, it can also ensure that the normal operation of cluster, and after the data center disconnected reconnects, it is ensured that the consistency of data between each data center.
Description
Technical field
The present invention relates to technical field of data processing more particularly to a kind of storage system building methods and device.
Background technique
Current storage system building method is mainly that a Mysql database, each number is arranged in each data center
Real-time synchronization is carried out according between supercentral Mysql database, to ensure the consistency of data between each data center.It is above-mentioned
In scheme, data are stored in the way of table in Mysql database, and every table requires to define major key, and only supports
InnoDB storage engines, and current alarm data does not all include generally major key, and can not necessarily InnoDB be supported to search for
Engine is difficult to use in the processing such as storage, inquiry to alarm data so as to cause above scheme.In addition, the realization of above scheme,
It needs network between each data center to keep stablizing, works as unstable networks, when some data center disconnects, be then difficult to really
Protect the consistency of data between each data center.
Summary of the invention
The present invention is directed to solve at least some of the technical problems in related technologies.
For this purpose, the first purpose of this invention is to propose a kind of storage system building method, for solving the prior art
In storage system be difficult to be suitable for alarm data, and the problem of be difficult to ensure data consistency between each data center.
Second object of the present invention is to propose that a kind of storage system builds device.
Third object of the present invention is to propose that another storage system builds device.
Fourth object of the present invention is to propose a kind of non-transitorycomputer readable storage medium.
5th purpose of the invention is to propose a kind of computer program product.
In order to achieve the above object, first aspect present invention embodiment proposes a kind of storage system building method, comprising:
Obtain the pending multiple data centers built；
At least two nodes of ES cluster are set in each data center, at least one master point is provided in the node
Piece, for storing data data at center；The data include: alarm configuration data and alarm data；
According to the node being arranged in each data center, the ES cluster in multiple data centers is built in generation.
Further, it is arranged in each data center after at least two nodes of ES cluster, further includes:
The first node of the first preset quantity is chosen from the node；
Copy fragment is set on the first node, the content of all main fragments is preserved in the copy fragment, with
When main fragment breaks down, the content of the main fragment of failure is refreshed using the content of the copy fragment.
Further, it is arranged in each data center after at least two nodes of ES cluster, further includes:
The second node is determined as candidate main section by the second node that the second preset quantity is chosen from the node
Point；
Determine the priority of each candidate host node；
In the starting of ES cluster, candidate host node without failure in each candidate host node is obtained, will not occur event
The candidate host node of highest priority is determined as current primary node in the candidate host node of barrier, so that the current primary node obtains
The content of last updated first authentic copy fragment brushes the content of each node according to the content of the first authentic copy fragment
Newly.
Further, described according to the node being arranged in each data center, generation is built in multiple data centers
After ES cluster, further includes:
Obtaining ES cluster is the interface that each data center provides, and the interface is used for node in corresponding data center
Content handled；
The interface is packaged, first interface is obtained, so that data center is in the first interface tune for receiving user
When with request, the content of node on own data center or other data centers is handled.
Further, the method further include:
Mysql database is set in each data center, meets preset condition for storing in the data center
First data, the preset condition include: that the format of first data meets the memory requirement of Mysql database, and/or,
Synchronizing frequency required by first data is greater than predeterminated frequency threshold value；
The ES cluster, for storing other data in the data center in addition to first data.
Further, the method further include:
It obtains ES cluster and Mysql database is the interface that each data center provides, the interface is used for corresponding
The content of node or the content of Mysql database are handled in data center；
The interface is packaged, second interface is obtained, so that data center is in the second interface tune for receiving user
When with request, the content of content or Mysql database to node on own data center or other data centers is carried out
Processing.
The storage system building method of the embodiment of the present invention, by obtaining the pending multiple data centers built；Every
At least two nodes of ES cluster are set in a data center, are provided at least one main fragment in node, for storing data
The data at center；Data include: alarm configuration data and alarm data；It is raw according to the node being arranged in each data center
At the ES cluster in multiple data centers is built, so as to be carried out using alarm data of the ES cluster to multiple data centers
The processing such as storage, inquiry, and even if some data center disconnects, it can also ensure that the normal operation of cluster, and disconnected
After the data center's reconnect for opening connection, it is ensured that the consistency of data between each data center.
In order to achieve the above object, second aspect of the present invention embodiment, which proposes a kind of storage system, builds device, comprising:
Module is obtained, for obtaining the pending multiple data centers built；
Setup module is arranged in the node at least two nodes of ES cluster to be arranged in each data center
There is at least one main fragment, for storing data the data at center；The data include: alarm configuration data and alarm number
According to；
Generation module, for according to the node being arranged in each data center, generation to be built in multiple data centers
ES cluster.
Further, the device further include: first chooses module；
Described first chooses module, for choosing the first node of the first preset quantity from the node；
The setup module is also used to be arranged on the first node copy fragment, preserves in the copy fragment
The content of all main fragments, to be divided the master of failure using the content of the copy fragment when main fragment breaks down
The content of piece is refreshed.
Further, the device further include: second chooses module, determining module and processing module；
Described second chooses module, for choosing the second node of the second preset quantity from the node, by described the
Two nodes are determined as candidate host node；
The determining module, for determining the priority of each candidate host node；
The processing module, in the starting of ES cluster, obtaining candidate without failure in each candidate host node
The candidate host node of highest priority in without failure candidate host node is determined as current primary node by host node, so that
The current primary node obtains the content of last updated first authentic copy fragment, according to the content of the first authentic copy fragment to each
The content of a node is refreshed.
Further, the device further include: the first package module；
The acquisition module, being also used to obtain ES cluster is the interface that provides of each data center, the interface for pair
The content of node is handled in corresponding data center；
First package module obtains first interface, so that data center is connecing for being packaged to the interface
When receiving the first interface call request of user, at the content of node on own data center or other data centers
Reason.
Further, the setup module is also used to that Mysql database is arranged in each data center, for storing
Meet the first data of preset condition in the data center, the preset condition includes: that the format of first data meets
The memory requirement of Mysql database, and/or, synchronizing frequency required by first data is greater than predeterminated frequency threshold value；
The ES cluster, for storing other data in the data center in addition to first data.
Further, the device further include: the second package module；
The acquisition module, is also used to obtain ES cluster and Mysql database is the interface that each data center provides, institute
Interface is stated for handling the content of node in corresponding data center or the content of Mysql database；
Second package module obtains second interface, so that data center is connecing for being packaged to the interface
When receiving the second interface call request of user, content to node on own data center or other data centers or
The content of Mysql database is handled.
The storage system of the embodiment of the present invention builds device, by obtaining the pending multiple data centers built；Every
At least two nodes of ES cluster are set in a data center, are provided at least one main fragment in node, for storing data
The data at center；Data include: alarm configuration data and alarm data；It is raw according to the node being arranged in each data center
At the ES cluster in multiple data centers is built, so as to be carried out using alarm data of the ES cluster to multiple data centers
The processing such as storage, inquiry, and even if some data center disconnects, it can also ensure that the normal operation of cluster, and disconnected
After the data center's reconnect for opening connection, it is ensured that the consistency of data between each data center.
In order to achieve the above object, third aspect present invention embodiment, which proposes another storage system, builds device, comprising: deposit
Reservoir, processor and storage are on a memory and the computer program that can run on a processor, which is characterized in that the processing
Device realizes storage system building method as described above when executing described program.
To achieve the goals above, fourth aspect present invention embodiment proposes a kind of computer readable storage medium,
On be stored with computer program, which realizes storage system building method as described above when being executed by processor.
To achieve the goals above, fifth aspect present invention embodiment proposes a kind of computer program product, when described
When instruction processing unit in computer program product executes, storage system building method as described above is realized.
The additional aspect of the present invention and advantage will be set forth in part in the description, and will partially become from the following description
Obviously, or practice through the invention is recognized.
Detailed description of the invention
Above-mentioned and/or additional aspect and advantage of the invention will become from the following description of the accompanying drawings of embodiments
Obviously and it is readily appreciated that, in which:
Fig. 1 is a kind of flow diagram of storage system building method provided in an embodiment of the present invention；
Fig. 2 is the structural schematic diagram that a kind of storage system provided in an embodiment of the present invention builds device；
Fig. 3 is the structural schematic diagram that another storage system provided in an embodiment of the present invention builds device；
Fig. 4 is the structural schematic diagram that another storage system provided in an embodiment of the present invention builds device；
Fig. 5 is the structural schematic diagram that another storage system provided in an embodiment of the present invention builds device；
Fig. 6 is the structural schematic diagram that another storage system provided in an embodiment of the present invention builds device；
Fig. 7 is the structural schematic diagram that another storage system provided in an embodiment of the present invention builds device.
Specific embodiment
The embodiment of the present invention is described below in detail, examples of the embodiments are shown in the accompanying drawings, wherein from beginning to end
Same or similar label indicates same or similar element or element with the same or similar functions.Below with reference to attached
The embodiment of figure description is exemplary, it is intended to is used to explain the present invention, and is not considered as limiting the invention.
Below with reference to the accompanying drawings the storage system building method and device of the embodiment of the present invention are described.
Fig. 1 is a kind of flow diagram of storage system building method provided in an embodiment of the present invention.As shown in Figure 1, should
Storage system building method the following steps are included:
S101, the pending multiple data centers built are obtained.
The executing subject of storage system building method provided by the invention is that storage system builds device, and storage system is built
Device can be the hardware devices such as server, or the software to install on hardware device.In the present embodiment, data center is for example
It can be for positioned at the computer room etc. of different location.Wherein, such as positioned at Pekinese's computer room, positioned at the computer room in Shanghai, positioned at Shenzhen
Computer room etc..
S102, at least two nodes that ES cluster is arranged in each data center, are provided at least one master in node
Fragment, for storing data data at center；Data include: alarm configuration data and alarm data.
In the present embodiment, ES (Elastic search) cluster is made of multiple nodes, and each node is equivalent to an ES
Example.Each node cluster.name having the same cooperates, sharing data and load.When be added new node or
When person deletes a node, ES cluster will be perceived, and be carried out to the fragment in each node balanced.
In the present embodiment, at least two nodes are set in each data center, so that in some data center's failure,
There are enough nodes in other data centers, ES cluster is run well.It, can be on each node in the present embodiment
Main fragment and copy fragment carry out equalization setting so that having enough in the case where the data center of half disconnects
Main fragment and copy fragment enable ES cluster to run well.
Further, on the basis of the above embodiments, after step 102, the method can also include following step
It is rapid: the first node of the first preset quantity is chosen from node；Copy fragment is set on first node, is saved in copy fragment
There is the content of all main fragments, with when main fragment breaks down, using the content of copy fragment to the main fragment of failure
Content refreshed.Wherein, by taking the quantity of data center is 6 as an example, the quantity of copy fragment can be 3 to 5, set respectively
It sets in 3 to 5 data centers, so that ES cluster is also can be according to pair in the case where half data center disconnects
This fragment carries out content recovery, it is ensured that ES cluster runs well.
Further, on the basis of the above embodiments, after step 102, the method can also include following step
It is rapid: to choose the second node of the second preset quantity from node, second node is determined as candidate host node；Determine each candidate
The priority of host node；In the starting of ES cluster, candidate host node without failure in each candidate host node is obtained, it will not
The candidate host node of highest priority is determined as current primary node in the candidate host node of failure, so that current primary node obtains
The content for taking last updated first authentic copy fragment is brushed according to content of the content of first authentic copy fragment to each node
Newly.
In the present embodiment, multiple candidate host nodes are chosen in advance, and the priority of each candidate host node is set, so that ES
After cluster starting, host node will not be randomly selected, and refresh according to content of the content of host node to other nodes, but
By without failure and highest priority candidate host node as current primary node, current primary node is according to last updated
The content of one copy fragment refreshes the content of each node.
S103, according to the node being arranged in each data center, the ES cluster in multiple data centers is built in generation.
Further, on the basis of the above embodiments, the method can be the following steps are included: obtain ES cluster
For the interface that each data center provides, interface is for handling the content of node in corresponding data center；To interface
It is packaged, obtains first interface, so that data center is in the first interface call request for receiving user, to data
The content of node is handled on center or other data centers.
Wherein, ES cluster be the interface that provides of data center include: increase interface, delete interface, modification interface, inquiry connect
Mouthful etc., increase processing, delete processing, modification processing, query processing etc. are carried out for the content to node in the data center.It is false
If ES cluster is that the interface quantity that each data center provides is 3, and the quantity of data center is n, then ES cluster is provided
Interface total quantity be 3n.After being packaged to interface, obtain 3 first interfaces, user by this 3 first interfaces into
Row calls, and may be implemented to carry out the content of node in multiple data centers increase processing, delete processing, modification processing, inquiry
Processing etc..
Further, on the basis of the above embodiments, the method is further comprising the steps of: in each data center
Upper setting Mysql database, meets the first data of preset condition for storing data in center, preset condition includes: first
The format of data meets the memory requirement of Mysql database, and/or, synchronizing frequency required by the first data is greater than default frequency
Rate threshold value；ES cluster, for storing data other data in center in addition to the first data.
Wherein, the format of the first data meets the memory requirement of Mysql database, such as can be, wraps in the first data
Major key containing table and the first data support InnoDB storage engines etc..
In the present embodiment, synchronism in data center is wanted, and/or, corresponding data format meets Mysql number
It stores according to the data of library memory requirement into Mysql database, can be improved the same part of the partial data, and Mysql data
Library can be suitable for handling storage, inquiry of the partial data etc..By other numbers in data center in addition to the first data
According to storage into ES cluster, it can be ensured that the consistency of data between each data center.
It is corresponding, in each data center be arranged Mysql database on the basis of, the method can also include with
Lower step: obtaining ES cluster and Mysql database is the interface that each data center provides, and interface is used for in corresponding data
The content of node or the content of Mysql database are handled in the heart；Interface is packaged, second interface is obtained, so that
Data center is in the second interface call request for receiving user, to node on own data center or other data centers
Content or the content of Mysql database handled.
Wherein, ES cluster be the interface that provides of data center include: increase interface, delete interface, modification interface, inquiry connect
Mouthful etc., increase processing, delete processing, modification processing, query processing etc. are carried out for the content to node in the data center.Separately
Outside, Mysql database also provides above-mentioned interface for data center, it is assumed that at this time provided by ES cluster and Mysql database
Interface total quantity is 2 × 3 × n=6n.After being packaged to interface, 3 second interfaces are obtained, user passes through to this 3
Two interfaces are called, and may be implemented to carry out the content of node in multiple data centers increase processing, delete processing, at modification
Reason, query processing etc., or increase processing, delete processing, modification processing, query processing are carried out to the content of Mysql database
Deng.
The storage system building method of the embodiment of the present invention, by obtaining the pending multiple data centers built；Every
At least two nodes of ES cluster are set in a data center, are provided at least one main fragment in node, for storing data
The data at center；Data include: alarm configuration data and alarm data；It is raw according to the node being arranged in each data center
At the ES cluster in multiple data centers is built, so as to be carried out using alarm data of the ES cluster to multiple data centers
The processing such as storage, inquiry, and even if some data center disconnects, it can also ensure that the normal operation of cluster, and disconnected
After the data center's reconnect for opening connection, it is ensured that the consistency of data between each data center.
Fig. 2 is the structural schematic diagram that a kind of storage system provided in an embodiment of the present invention builds device.As shown in Fig. 2, packet
It includes: obtaining module 21, setup module 22 and generation module 23.
Wherein, module 21 is obtained, for obtaining the pending multiple data centers built；
Setup module 22 is set in the node at least two nodes of ES cluster to be arranged in each data center
It is equipped at least one main fragment, for storing data the data at center；The data include: alarm configuration data and alarm number
According to；
Generation module 23, for according to the node being arranged in each data center, generation to be built in multiple data centers
ES cluster.
Storage system provided by the invention, which builds device, to be the hardware devices such as server, or to pacify on hardware device
The software of dress.In the present embodiment, data center for example can be for positioned at the computer room etc. of different location.Wherein, such as positioned at Beijing
Computer room, the computer room positioned at Shanghai, the computer room positioned at Shenzhen etc..
In the present embodiment, ES (Elastic search) cluster is made of multiple nodes, and each node is equivalent to an ES
Example.Each node cluster.name having the same cooperates, sharing data and load.When be added new node or
When person deletes a node, ES cluster will be perceived, and be carried out to the fragment in each node balanced.
In the present embodiment, at least two nodes are set in each data center, so that in some data center's failure,
There are enough nodes in other data centers, ES cluster is run well.It, can be on each node in the present embodiment
Main fragment and copy fragment carry out equalization setting so that having enough in the case where the data center of half disconnects
Main fragment and copy fragment enable ES cluster to run well.
Further, in conjunction with reference Fig. 3, on the basis of embodiment shown in Fig. 2, the device can also include:
One chooses module 24；
Wherein, described first module 24 is chosen, for choosing the first node of the first preset quantity from the node；
The setup module 22 is also used to be arranged on the first node copy fragment, saves in the copy fragment
There is the content of all main fragments, with when main fragment breaks down, using the content of the copy fragment to the master of failure
The content of fragment is refreshed.
Wherein, by taking the quantity of data center is 6 as an example, the quantity of copy fragment can be 3 to 5, be separately positioned on 3 and arrive
In 5 data centers, thus in the case where half data center disconnects, ES cluster be also can according to copy fragment into
Row content recovery, it is ensured that ES cluster runs well.
Further, in conjunction with reference Fig. 4, on the basis of embodiment shown in Fig. 2, the device can also include:
Two choose module 25, determining module 26 and processing module 27；
Described second chooses module 25, will be described for choosing the second node of the second preset quantity from the node
Second node is determined as candidate host node；
The determining module 26, for determining the priority of each candidate host node；
The processing module 27, in the starting of ES cluster, obtaining time without failure in each candidate host node
Host node is selected, the candidate host node of highest priority in without failure candidate host node is determined as current primary node, with
The current primary node is set to obtain the content of last updated first authentic copy fragment, according to the content pair of the first authentic copy fragment
The content of each node is refreshed.
In the present embodiment, multiple candidate host nodes are chosen in advance, and the priority of each candidate host node is set, so that ES
After cluster starting, host node will not be randomly selected, and refresh according to content of the content of host node to other nodes, but
By without failure and highest priority candidate host node as current primary node, current primary node is according to last updated
The content of one copy fragment refreshes the content of each node.
Further, on the basis of the above embodiments, in conjunction with reference Fig. 5, on the basis of embodiment shown in Fig. 2, institute
The device stated can also include: the first package module 28；
Wherein, the acquisition module 21, being also used to obtain ES cluster is the interface that each data center provides, the interface
It is handled for the content to node in corresponding data center；
First package module 28 obtains first interface, so that data center exists for being packaged to the interface
When receiving the first interface call request of user, the content of node on own data center or other data centers is carried out
Processing.
Wherein, ES cluster be the interface that provides of data center include: increase interface, delete interface, modification interface, inquiry connect
Mouthful etc., increase processing, delete processing, modification processing, query processing etc. are carried out for the content to node in the data center.It is false
If ES cluster is that the interface quantity that each data center provides is 3, and the quantity of data center is n, then ES cluster is provided
Interface total quantity be 3n.After being packaged to interface, obtain 3 first interfaces, user by this 3 first interfaces into
Row calls, and may be implemented to carry out the content of node in multiple data centers increase processing, delete processing, modification processing, inquiry
Processing etc..
Further, on the basis of the above embodiments, the setup module 22 is also used to set in each data center
Mysql database is set, for storing the first data for meeting preset condition in the data center, the preset condition includes:
The format of first data meets the memory requirement of Mysql database, and/or, synchronizing band required by first data
Rate is greater than predeterminated frequency threshold value；The ES cluster, for storing other in the data center in addition to first data
Data.
Wherein, the format of the first data meets the memory requirement of Mysql database, such as can be, wraps in the first data
Major key containing table and the first data support InnoDB storage engines etc..
In the present embodiment, synchronism in data center is wanted, and/or, corresponding data format meets Mysql number
It stores according to the data of library memory requirement into Mysql database, can be improved the same part of the partial data, and Mysql data
Library can be suitable for handling storage, inquiry of the partial data etc..By other numbers in data center in addition to the first data
According to storage into ES cluster, it can be ensured that the consistency of data between each data center.
It is corresponding, it is shown in Fig. 2 in conjunction with reference Fig. 6 on the basis of Mysql database is set in each data center
On the basis of embodiment, it can also include: the second package module 29 that storage system, which builds device,；
The acquisition module 21, is also used to obtain ES cluster and Mysql database is the interface that each data center provides,
The interface is for handling the content of node in corresponding data center or the content of Mysql database；
Second package module 29 obtains second interface, so that data center exists for being packaged to the interface
When receiving the second interface call request of user, content to node on own data center or other data centers or
The content of Mysql database is handled.
Wherein, ES cluster be the interface that provides of data center include: increase interface, delete interface, modification interface, inquiry connect
Mouthful etc., increase processing, delete processing, modification processing, query processing etc. are carried out for the content to node in the data center.Separately
Outside, Mysql database also provides above-mentioned interface for data center, it is assumed that at this time provided by ES cluster and Mysql database
Interface total quantity is 2 × 3 × n=6n.After being packaged to interface, 3 second interfaces are obtained, user passes through to this 3
Two interfaces are called, and may be implemented to carry out the content of node in multiple data centers increase processing, delete processing, at modification
Reason, query processing etc., or increase processing, delete processing, modification processing, query processing are carried out to the content of Mysql database
Deng.
The storage system of the embodiment of the present invention builds device, by obtaining the pending multiple data centers built；Every
At least two nodes of ES cluster are set in a data center, are provided at least one main fragment in node, for storing data
The data at center；Data include: alarm configuration data and alarm data；It is raw according to the node being arranged in each data center
At the ES cluster in multiple data centers is built, so as to be carried out using alarm data of the ES cluster to multiple data centers
The processing such as storage, inquiry, and even if some data center disconnects, it can also ensure that the normal operation of cluster, and disconnected
After the data center's reconnect for opening connection, it is ensured that the consistency of data between each data center.
Fig. 7 is the structural schematic diagram that another storage system provided in an embodiment of the present invention builds device.The storage system
Building device includes:
Memory 1001, processor 1002 and it is stored in the calculating that can be run on memory 1001 and on processor 1002
Machine program.
Processor 1002 realizes the storage system building method provided in above-described embodiment when executing described program.
Further, storage system builds device further include:
Communication interface 1003, for the communication between memory 1001 and processor 1002.
Memory 1001, for storing the computer program that can be run on processor 1002.
Memory 1001 may include high speed RAM memory, it is also possible to further include nonvolatile memory (non-
Volatile memory), a for example, at least magnetic disk storage.
Processor 1002 realizes storage system building method described in above-described embodiment when for executing described program.
If memory 1001, processor 1002 and the independent realization of communication interface 1003, communication interface 1003, memory
1001 and processor 1002 can be connected with each other by bus and complete mutual communication.The bus can be industrial standard
Architecture (Industry Standard Architecture, referred to as ISA) bus, external equipment interconnection
(Peripheral Component, referred to as PCI) bus or extended industry-standard architecture (Extended Industry
Standard Architecture, referred to as EISA) bus etc..The bus can be divided into address bus, data/address bus, control
Bus processed etc..Only to be indicated with a thick line in Fig. 7, it is not intended that an only bus or a type of convenient for indicating
Bus.
Optionally, in specific implementation, if memory 1001, processor 1002 and communication interface 1003, are integrated in one
It is realized on block chip, then memory 1001, processor 1002 and communication interface 1003 can be completed mutual by internal interface
Communication.
Processor 1002 may be a central processing unit (Central Processing Unit, referred to as CPU), or
Person is specific integrated circuit (Application Specific Integrated Circuit, referred to as ASIC) or quilt
It is configured to implement one or more integrated circuits of the embodiment of the present invention.
The present invention also provides a kind of non-transitorycomputer readable storage mediums, are stored thereon with computer program, the journey
Storage system building method as described above is realized when sequence is executed by processor.
The present invention also provides a kind of computer program products, when the instruction processing unit in the computer program product executes
When, realize storage system building method as described above.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or spy described in conjunction with this embodiment or example
Point is included at least one embodiment or example of the invention.In the present specification, schematic expression of the above terms are not
It must be directed to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be in office
It can be combined in any suitable manner in one or more embodiment or examples.In addition, without conflicting with each other, the skill of this field
Art personnel can tie the feature of different embodiments or examples described in this specification and different embodiments or examples
It closes and combines.
In addition, term " first ", " second " are used for descriptive purposes only and cannot be understood as indicating or suggesting relative importance
Or implicitly indicate the quantity of indicated technical characteristic.Define " first " as a result, the feature of " second " can be expressed or
Implicitly include at least one this feature.In the description of the present invention, the meaning of " plurality " is at least two, such as two, three
It is a etc., unless otherwise specifically defined.
Any process described otherwise above or method description are construed as in flow chart or herein, and expression includes
It is one or more for realizing custom logic function or process the step of executable instruction code module, segment or portion
Point, and the range of the preferred embodiment of the present invention includes other realization, wherein can not press shown or discussed suitable
Sequence, including according to related function by it is basic simultaneously in the way of or in the opposite order, Lai Zhihang function, this should be of the invention
Embodiment person of ordinary skill in the field understood.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use
In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (such as computer based system, including the system of processor or other can be held from instruction
The instruction fetch of row system, device or equipment and the system executed instruction) it uses, or combine these instruction execution systems, device or set
It is standby and use.For the purpose of this specification, " computer-readable medium ", which can be, any may include, stores, communicates, propagates or pass
Defeated program is for instruction execution system, device or equipment or the dress used in conjunction with these instruction execution systems, device or equipment
It sets.The more specific example (non-exhaustive list) of computer-readable medium include the following: there is the electricity of one or more wirings
Interconnecting piece (electronic device), portable computer diskette box (magnetic device), random access memory (RAM), read-only memory
(ROM), erasable edit read-only storage (EPROM or flash memory), fiber device and portable optic disk is read-only deposits
Reservoir (CDROM).In addition, computer-readable medium can even is that the paper that can print described program on it or other are suitable
Medium, because can then be edited, be interpreted or when necessary with it for example by carrying out optical scanner to paper or other media
His suitable method is handled electronically to obtain described program, is then stored in computer storage.
It should be appreciated that each section of the invention can be realized with hardware, software, firmware or their combination.Above-mentioned
In embodiment, software that multiple steps or method can be executed in memory and by suitable instruction execution system with storage
Or firmware is realized.Such as, if realized with hardware in another embodiment, following skill well known in the art can be used
Any one of art or their combination are realized: have for data-signal is realized the logic gates of logic function from
Logic circuit is dissipated, the specific integrated circuit with suitable combinational logic gate circuit, programmable gate array (PGA), scene can compile
Journey gate array (FPGA) etc..
Those skilled in the art are understood that realize all or part of step that above-described embodiment method carries
It suddenly is that relevant hardware can be instructed to complete by program, the program can store in a kind of computer-readable storage medium
In matter, which when being executed, includes the steps that one or a combination set of embodiment of the method.
It, can also be in addition, each functional unit in each embodiment of the present invention can integrate in a processing module
It is that each unit physically exists alone, can also be integrated in two or more units in a module.Above-mentioned integrated mould
Block both can take the form of hardware realization, can also be realized in the form of software function module.The integrated module is such as
Fruit is realized and when sold or used as an independent product in the form of software function module, also can store in a computer
In read/write memory medium.
Storage medium mentioned above can be read-only memory, disk or CD etc..Although having been shown and retouching above
The embodiment of the present invention is stated, it is to be understood that above-described embodiment is exemplary, and should not be understood as to limit of the invention
System, those skilled in the art can be changed above-described embodiment, modify, replace and become within the scope of the invention
Type.
Claims (15)
1. a kind of storage system building method characterized by comprising
Obtain the pending multiple data centers built；
At least two nodes of ES cluster are set in each data center, are provided at least one main fragment in the node,
The data at center for storing data；The data include: alarm configuration data and alarm data；
According to the node being arranged in each data center, the ES cluster in multiple data centers is built in generation.
2. the method according to claim 1, wherein at least two of ES cluster is arranged in each data center
After node, further includes:
The first node of the first preset quantity is chosen from the node；
Copy fragment is set on the first node, the content of all main fragments is preserved in the copy fragment, in master
When fragment breaks down, the content of the main fragment of failure is refreshed using the content of the copy fragment.
3. the method according to claim 1, wherein at least two of ES cluster is arranged in each data center
After node, further includes:
The second node is determined as candidate host node by the second node that the second preset quantity is chosen from the node；
Determine the priority of each candidate host node；
In the starting of ES cluster, candidate host node without failure in each candidate host node is obtained, it will be without failure
The candidate host node of highest priority is determined as current primary node in candidate host node, so that current primary node acquisition is newest
The content of the first authentic copy fragment of update refreshes the content of each node according to the content of the first authentic copy fragment.
4. the method according to claim 1, wherein described according to the node being arranged in each data center, life
At building after the ES cluster in multiple data centers, further includes:
Obtaining ES cluster is the interface that each data center provides, and the interface is used for in node in corresponding data center
Appearance is handled；
The interface is packaged, first interface is obtained, so that data center asks in the first interface calling for receiving user
When asking, the content of node on own data center or other data centers is handled.
5. the method according to claim 1, wherein further include:
Mysql database is set in each data center, meets the first of preset condition in the data center for storing
Data, the preset condition include: that the format of first data meets the memory requirement of Mysql database, and/or, it is described
Synchronizing frequency required by first data is greater than predeterminated frequency threshold value；
The ES cluster, for storing other data in the data center in addition to first data.
6. according to the method described in claim 5, it is characterized by further comprising:
It obtains ES cluster and Mysql database is the interface that each data center provides, the interface is used for corresponding data
The content of node or the content of Mysql database are handled on center；
The interface is packaged, second interface is obtained, so that data center asks in the second interface calling for receiving user
When asking, the content of node on own data center or other data centers or the content of Mysql database are handled.
7. a kind of storage system builds device characterized by comprising
Module is obtained, for obtaining the pending multiple data centers built；
Setup module, be provided at least two nodes of ES cluster to be arranged in each data center, in the node to
Lack a main fragment, for storing data the data at center；The data include: alarm configuration data and alarm data；
Generation module, for according to the node being arranged in each data center, the ES collection in multiple data centers to be built in generation
Group.
8. device according to claim 7, which is characterized in that further include: first chooses module；
Described first chooses module, for choosing the first node of the first preset quantity from the node；
The setup module is also used to be arranged on the first node copy fragment, preserves in the copy fragment all
The content of main fragment, with when main fragment breaks down, using the content of the copy fragment to the main fragment of failure
Content is refreshed.
9. device according to claim 7, which is characterized in that further include: second chooses module, determining module and processing mould
Block；
Described second chooses module, for choosing the second node of the second preset quantity from the node, by second section
Point is determined as candidate host node；
The determining module, for determining the priority of each candidate host node；
The processing module, in the starting of ES cluster, obtaining the main section of candidate without failure in each candidate host node
The candidate host node of highest priority in without failure candidate host node is determined as current primary node, so that described by point
Current primary node obtains the content of last updated first authentic copy fragment, according to the content of the first authentic copy fragment to each section
The content of point is refreshed.
10. device according to claim 7, which is characterized in that further include: the first package module；
The acquisition module, being also used to obtain ES cluster is the interface that each data center provides, and the interface is used for corresponding
Data center on the content of node handled；
First package module obtains first interface, so that data center is receiving for being packaged to the interface
When the first interface call request of user, the content of node on own data center or other data centers is handled.
11. device according to claim 7, which is characterized in that the setup module is also used in each data center
Mysql database is set, for storing the first data for meeting preset condition in the data center, the preset condition packet
Include: the format of first data meets the memory requirement of Mysql database, and/or, synchronization required by first data
Frequency is greater than predeterminated frequency threshold value；
The ES cluster, for storing other data in the data center in addition to first data.
12. device according to claim 11, which is characterized in that further include: the second package module；
The acquisition module, is also used to obtain ES cluster and Mysql database is the interface that each data center provides, described to connect
Mouth is for handling the content of node in corresponding data center or the content of Mysql database；
Second package module obtains second interface, so that data center is receiving for being packaged to the interface
When the second interface call request of user, to the content or Mysql of node on own data center or other data centers
The content of database is handled.
13. a kind of storage system builds device characterized by comprising
Memory, processor and storage are on a memory and the computer program that can run on a processor, which is characterized in that institute
It states when processor executes described program and realizes such as storage system building method as claimed in any one of claims 1 to 6.
14. a kind of non-transitorycomputer readable storage medium, is stored thereon with computer program, which is characterized in that the program
Such as storage system building method as claimed in any one of claims 1 to 6 is realized when being executed by processor.
15. a kind of computer program product realizes such as right when the instruction processing unit in the computer program product executes
It is required that any storage system building method in 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910072199.1A CN109840266A (en) | 2019-01-25 | 2019-01-25 | Storage system building method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910072199.1A CN109840266A (en) | 2019-01-25 | 2019-01-25 | Storage system building method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109840266A true CN109840266A (en) | 2019-06-04 |
Family
ID=66884130
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910072199.1A Pending CN109840266A (en) | 2019-01-25 | 2019-01-25 | Storage system building method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109840266A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112866386A (en) * | 2021-01-19 | 2021-05-28 | 青岛越超传媒有限公司 | Data storage data construction method based on cloud computing |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106776878A (en) * | 2016-11-29 | 2017-05-31 | 西安交通大学 | A kind of method for carrying out facet retrieval to MOOC courses based on ElasticSearch |
| CN107329987A (en) * | 2017-06-02 | 2017-11-07 | 广州启辰电子科技有限公司 | A kind of search system based on mongo databases |
| CN108446367A (en) * | 2018-03-15 | 2018-08-24 | 湖南工业大学 | A kind of the packaging industry data search method and equipment of knowledge based collection of illustrative plates |
| US20180268000A1 (en) * | 2017-03-20 | 2018-09-20 | Datameer, Inc. | Apparatus and Method for Distributed Query Processing Utilizing Dynamically Generated In-Memory Term Maps |
| US20190005029A1 (en) * | 2017-06-30 | 2019-01-03 | Jpmorgan Chase Bank, N.A. | Systems and methods for natural language processing of structured documents |
-
2019
- 2019-01-25 CN CN201910072199.1A patent/CN109840266A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106776878A (en) * | 2016-11-29 | 2017-05-31 | 西安交通大学 | A kind of method for carrying out facet retrieval to MOOC courses based on ElasticSearch |
| US20180268000A1 (en) * | 2017-03-20 | 2018-09-20 | Datameer, Inc. | Apparatus and Method for Distributed Query Processing Utilizing Dynamically Generated In-Memory Term Maps |
| CN107329987A (en) * | 2017-06-02 | 2017-11-07 | 广州启辰电子科技有限公司 | A kind of search system based on mongo databases |
| US20190005029A1 (en) * | 2017-06-30 | 2019-01-03 | Jpmorgan Chase Bank, N.A. | Systems and methods for natural language processing of structured documents |
| CN108446367A (en) * | 2018-03-15 | 2018-08-24 | 湖南工业大学 | A kind of the packaging industry data search method and equipment of knowledge based collection of illustrative plates |
Non-Patent Citations (2)
| Title |
|---|
| 化立志: ""基于知识图谱的领域知识库管理系统的设计与实现"", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
| 曾亚飞: ""基于Elasticsearch的分布式智能搜索引擎的研究与实现"", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112866386A (en) * | 2021-01-19 | 2021-05-28 | 青岛越超传媒有限公司 | Data storage data construction method based on cloud computing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104935654B (en) | Caching method, write-in point client in a kind of server cluster system and read client | |
| CN103684839B (en) | It is a kind of for the data transmission method of two-node cluster hot backup, system and server | |
| CN105989076A (en) | Data statistical method and device | |
| CN103885865B (en) | A kind of Method of Sensor Management and device | |
| CN108897628A (en) | A kind of implementation method of distributed lock, device and electronic equipment | |
| CN106446159B (en) | A kind of method of storage file, the first virtual machine and name node | |
| CN106506703A (en) | Based on the service discovery method of shared drive, apparatus and system, server | |
| CN108287886A (en) | The method and device of synchrodata modification information | |
| CN109547512A (en) | A kind of method and device of the distributed Session management based on NoSQL | |
| CN110334094B (en) | Data query method, system, device and equipment based on inverted index | |
| CN106130972B (en) | resource access control method and device | |
| CN104054076B (en) | Date storage method, database purchase node failure processing method and processing device | |
| CN104301990B (en) | A kind of method and device of data syn-chronization | |
| CN108363641A (en) | A kind of standby machine data transferring method, control node and Database Systems | |
| CN103701653B (en) | The processing method of a kind of interface hot plug configuration data and network configuration server | |
| CN110008246A (en) | Metadata management method and device | |
| CN106503008A (en) | File memory method and device and file polling method and apparatus | |
| CN109840266A (en) | Storage system building method and device | |
| CN110297822A (en) | Key management method, device, equipment and storage medium towards block chain | |
| CN110059026A (en) | A kind of catalogue processing method, device and storage system | |
| CN104216836B (en) | A kind of concurrent reading and concurrent writing method and apparatus of storage system | |
| CN109857344A (en) | Heart beat status judgment method, device and computer equipment based on shared drive | |
| CN114579506A (en) | Inter-processor communication method, system, storage medium, and processor | |
| CN110874382B (en) | Data writing method, device and equipment thereof | |
| CN111209304B (en) | Data processing method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190604 |