CN109284294A - Acquire method and device, the storage medium, processor of data - Google Patents
Acquire method and device, the storage medium, processor of data Download PDFInfo
- Publication number
- CN109284294A CN109284294A CN201811190166.9A CN201811190166A CN109284294A CN 109284294 A CN109284294 A CN 109284294A CN 201811190166 A CN201811190166 A CN 201811190166A CN 109284294 A CN109284294 A CN 109284294A
- Authority
- CN
- China
- Prior art keywords
- host node
- node
- data
- target data
- intermediate station
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/80—Database-specific techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of method and devices for acquiring data, storage medium, processor.Wherein, this method comprises: the host node in device clusters issues acquisition request, wherein device clusters include at least: host node, from node, is the backup of host node from node at least one；Based on relational data structure, obtain acquisition and request requested target data, wherein relational data structure includes at least: the corresponding relationship between host node and communication intermediate station communicates the corresponding relationship between intermediate station and target data.The present invention solves the technical problem that collecting efficiency is low, the ability to ward off risks is poor when being written and read in the prior art using relevant database to mass data.
Description
Technical field
The present invention relates to data collecting fields, are situated between in particular to a kind of method and device for acquiring data, storage
Matter, processor.
Background technique
In the epoch of intelligence manufacture, the application of automation equipment improves the efficiency of plant produced, and incident is control
The acquisition and analysis and utilization problem for the magnanimity creation data that device processed generates.
Data acquisition platform currently on the market is all data storage to be carried out using relevant database, and industry is supervised mostly
Control data demand acquisition speed and response speed are Milliseconds, and a large enterprise is frequently present of tens of thousands of or even hundreds of thousands prison
Measuring point, if storing the high-frequency data of such large capacity with relational database, due to the theory that relevant database itself designs,
It is caused to be difficult to carry out the read-write operation of the data of hundreds of thousands per second.
Meanwhile conventional data collection system by server disposition in local, do not have the ability to ward off risks, once server
Delay machine or suddenly power-off will lose mass data before system recovery, this is to some higher productions for requiring data integrity
System is unacceptable.
In addition, communications protocol provided by each difference controller manufacturer is different, incompatible, a life is acquired
The data of the distinct device of producing line will often develop multiple kinds parsing module, so that the development efficiency of upper layer application is significantly
It reduces.
For above-mentioned problem, currently no effective solution has been proposed.
Summary of the invention
The embodiment of the invention provides a kind of method and devices for acquiring data, storage medium, processor, at least to solve
The skill that collecting efficiency is low, the ability to ward off risks is poor when being written and read in the prior art using relevant database to mass data
Art problem.
According to an aspect of an embodiment of the present invention, a kind of method for acquiring data is provided, comprising: in device clusters
Host node issues acquisition request, wherein device clusters include at least: host node and at least one from node, save based on the node
The backup of point；Based on relational data structure, obtains acquisition and request requested target data, wherein relational data structure is at least
Include: the corresponding relationship between host node and communication intermediate station, communicates the corresponding relationship between intermediate station and target data.
Optionally, communication intermediate station is for converting target data format.
Optionally, after host node is acquired target data, comprising: host node interval preset time is by write instruction
It is saved in AOF file.
Optionally, after write instruction is saved in AOF file by host node interval preset time, comprising: host node will
The write instruction that same data carry out is merged in AOF file, obtains merging file；Host node will merge file backup at least
One from node.
Another aspect according to an embodiment of the present invention provides a kind of method for acquiring data, comprising: obtain number of targets
According to the first corresponding relationship between corresponding device and communication intermediate station, and obtain between the host node in communication intermediate station and cluster
The second corresponding relationship；If getting the corresponding relationship between host node and target data corresponding device, host node acquires mesh
Mark data, wherein include at least host node in cluster at least one from node, be the backup of host node from node.
Optionally, before host node acquisition target data, comprising: establish host node and communication according to IP address and port numbers
Corresponding relationship between intermediate station, communication intermediate station and target data corresponding device, wherein communication intermediate station is used for number of targets
It is converted according to format.
Optionally, communication intermediate station is for converting target data format, comprising: converts the format of target data
For key-value pair format.
Optionally, after host node acquisition target data, comprising: whether detection host node breaks down；It is sent out in host node
In the case where raw failure, any one in cluster is promoted to host node from node；After receiving redirection instruction, according to finger
Communication intermediate station is connected to new host node, redirects instruction for that will communicate intermediate station by IP address and port numbers in order
It is connected to new host node.
Optionally, after host node acquisition target data, comprising: write instruction is saved in AOF text by interval preset time
In part.
Optionally, interval preset time write instruction is saved in AOF file after, comprising: by AOF file to same
The write instruction that one data carry out merges, and obtains merging file；By merging file backup at least one from node.
Optionally, after write operation is saved in AOF file by interval preset time, comprising: needed in the data of node
When restoring, all instructions that AOF file saves is executed.
Other side according to an embodiment of the present invention provides a kind of device for acquiring data, comprising: transmission unit,
Issue acquisition request for the host node in device clusters, wherein device clusters include at least: host node and at least one from section
Point is the backup of host node from node；Acquiring unit obtains acquisition and requests requested mesh for being based on relational data structure
Mark data, wherein relational data structure includes at least: the corresponding relationship between host node and communication intermediate station communicates intermediate station
Corresponding relationship between target data.
Another aspect according to an embodiment of the present invention provides a kind of device for acquiring data, comprising: acquiring unit,
For obtaining target data corresponding device and communicating the first corresponding relationship between intermediate station, and obtain communication intermediate station and cluster
In host node between the second corresponding relationship；Acquisition unit, if for getting host node and target data corresponding device
Between corresponding relationship, host node acquires target data, wherein included at least in cluster host node and at least one from node,
It is the backup of host node from node.
Another aspect according to an embodiment of the present invention, provides a kind of storage medium, and storage medium includes the journey of storage
Sequence, wherein the method that equipment where control storage medium executes above-mentioned acquisition data in program operation.
Another aspect according to an embodiment of the present invention provides a kind of processor, and processor is used to run program,
In, program run when execute above-mentioned acquisition data method.
In embodiments of the present invention, using the Distributed Real Time Data Acquisition system of Redis cluster, by disposing Redis
Distributed type assemblies issue acquisition request by the host node in device clusters, wherein equipment in conjunction with the mode of OPC UA technology
Cluster includes at least: host node, from node, is the backup of host node from node at least one；Based on relational data structure, obtain
Acquisition is taken to request requested target data, wherein relational data structure includes at least: between host node and communication intermediate station
Corresponding relationship communicates the corresponding relationship between intermediate station and target data, has reached the mesh of the real-time read-write operation of mass data
, to realize the technical effect for improving system data acquisition efficiency and the ability to ward off risks, and then solve the prior art
The middle technical problem that collecting efficiency is low, the ability to ward off risks is poor when being written and read using relevant database to mass data.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present invention, constitutes part of this application, this hair
Bright illustrative embodiments and their description are used to explain the present invention, and are not constituted improper limitations of the present invention.In the accompanying drawings:
Fig. 1 is a kind of flow chart of method for acquiring data according to an embodiment of the present invention；
Fig. 2 is a kind of schematic diagram of the method for optional acquisition data according to an embodiment of the present invention；
Fig. 3 is the flow chart of the method for another acquisition data according to an embodiment of the present invention；
Fig. 4 is a kind of structure chart of device for acquiring data according to an embodiment of the present invention；
Fig. 5 is the structure chart of the device of another acquisition data according to an embodiment of the present invention.
Specific embodiment
In order to enable those skilled in the art to better understand the solution of the present invention, below in conjunction in the embodiment of the present invention
Attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is only
The embodiment of a part of the invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
The model that the present invention protects all should belong in member's every other embodiment obtained without making creative work
It encloses.
It should be noted that description and claims of this specification and term " first " in above-mentioned attached drawing, "
Two " etc. be to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should be understood that using in this way
Data be interchangeable under appropriate circumstances, so as to the embodiment of the present invention described herein can in addition to illustrating herein or
Sequence other than those of description is implemented.In addition, term " includes " and " having " and their any deformation, it is intended that cover
Cover it is non-exclusive include, for example, the process, method, system, product or equipment for containing a series of steps or units are not necessarily limited to
Step or unit those of is clearly listed, but may include be not clearly listed or for these process, methods, product
Or other step or units that equipment is intrinsic.
Caused with the high-frequency data of relational database storage large capacity due to the theory that relevant database itself designs
It is difficult to carry out the read-write operation of the data of hundreds of thousands per second.Conventional data collection system by server disposition in local, no
Have the ability to ward off risks, once server delay machine or suddenly power-off, will lose mass data before system recovery.Each different controls
Communications protocol provided by Zhi Qi manufacturer is different, incompatible, and the data that acquire the distinct device of a production line are past
Toward to develop multiple kinds parsing module, so that the development efficiency of upper layer application substantially reduces.
To solve the above problems, the embodiment of the present application provides corresponding solution, it is described further below.
According to embodiments of the present invention, a kind of embodiment of the method for acquiring data is provided, it should be noted that in attached drawing
The step of process illustrates can execute in a computer system such as a set of computer executable instructions, although also,
Logical order is shown in flow chart, but in some cases, it can be to be different from shown by sequence execution herein or retouch
The step of stating.
Fig. 1 is a kind of flow chart of method for acquiring data according to an embodiment of the present invention, as shown in Figure 1, this method packet
Include following steps:
Step S102 obtains target data corresponding device and communicates the first corresponding relationship between intermediate station, and obtains logical
Interrogate the second corresponding relationship between the host node in intermediate station and cluster；
Host node is established according to IP address and port numbers and communicates the first corresponding relationship of intermediate station, according to IP address and end
Slogan establish communication intermediate station and target data corresponding device between the second corresponding relationship, wherein communication intermediate station for pair
Target data format is converted.It will be in the first corresponding relationship and communication between target data corresponding device and communication intermediate station
Between stand and target data corresponding device between the second corresponding relationship be stored in XML configuration file.Through the above steps, will collect
Host node and target data in group establish corresponding relationship, and different host nodes has and the data pair in different PLC controllers
Answer, when acquiring data, host node according in XML configuration file IP address and port numbers acquire corresponding PLC and control
Data in device.For example, host node 1 is corresponding with PLC controller 1,2,3, host node 2 is corresponding with PLC controller 4, host node 3 with
PLC controller 5,6 is corresponding, and when carrying out data acquisition, host node 1 acquires the target data in PLC controller 1,2,3, main section
Target data in 2 acquisition PLC controller 4 of point, host node 3 acquire the target data in PLC controller 5,6.Communicate intermediate station
Including at least one of: interface server, OPC UA client, OPC UA server.Intermediate station is communicated by target data
Format is converted to key-value pair format (also known as Key-Value format).
The application provides a kind of optional embodiment: the PLC label point acquired will be needed to be configured to the address OPC UA browsing
Space file, selects PLC type number, configuration of IP address port number, and starting OPC UA server and PLC establish TCP connection.Pass through
OPC UA client can collect PLC label point data.Development interface server, by the OPC UA normal data lattice after parsing
Formula is automatically converted to the Key-Value format of Redis storage.
Step S104, if getting the corresponding relationship between host node and target data, host node acquires target data,
Wherein, host node is included at least in cluster at least one from node, is the backup of host node from node.
In the embodiment of the present application, each node in cluster and other nodes establish communication connection, which can
To be TCP communication.Host node is used for the acquisition and storage to data, from the node for the duplication that node is host node.Pass through node
Between establish TCP and communicate the monitoring that can achieve host node to other host nodes, host node detects whether other host nodes occur
Failure.
In the embodiment of the present application, it can also detect whether host node breaks down；The case where host node breaks down
Under, any one in cluster is promoted to host node from node；After receiving redirection instruction, according to the IP address in instruction
And port numbers, communication intermediate station is connected to new host node, instruction is redirected for intermediate station will to be communicated and is connected to new master
Node.
In the embodiment of the present application, cluster can also deposit data in addition to being acquired in the above process to data
Write instruction is saved in AOF file by storage, host node interval preset time.Write instruction is saved in the mistake in AOF file
Journey is AOF persistence or AOF mode, wherein AOF mode needs to be arranged in advance.Record Redis cluster host node writes behaviour
Make, time interval can be set to 1 second, when Redis cluster host node accident delay machine can restore number by executing AOF file
According at most only losing 1 second data before delay machine, loss be preferably minimized.
In the embodiment of the present application, the write instruction that same data carry out can also will be merged in AOF file, is closed
And file；By merging file backup at least one from node.When the data of node need to restore, executes AOF file and protect
The all instructions deposited.Above-mentioned will merge in AOF file to the write instruction of same data can prevent AOF file excessive.
The application provides a kind of optional embodiment, as shown in Fig. 2, by development deployment Redis distributed type assemblies, knot
It closes OPC UA and manages label point, realize factory's mass data collection, analysis, storage.Implementation method is as follows:
Firstly, configuring Redis main and subordinate node according to IP address and port numbers, TCP communication is established to each other, wherein host node
It for storing data, and is then the duplicate of some host node from node.It, can when user needs to handle more read requests
To add from node with the reading performance of expansion system.
In this optional embodiment, cluster carrys out the capacity of extending database using fragment, and the load of command request is handed over
It is shared to different nodes.Entire database is divided into 16384 slots (slot) by cluster, and all keys belong to this 16384
One of them of slot, the formula which slot calculation key (key) belongs to is slot_number=crc16 (key) %16384, wherein
The cyclic redundancy check and function that crc16 is 16.Each host node in cluster can handle 0 to 16384 slot, when
16384 slots have some node when being responsible for processing, and cluster enters online state, and start to process the data of client transmission
Command request.Thereafter, each slot is assigned host node and is monitored to it, and key can indicate target data.Client hair
The data command requests sent can be the read-write requests of data.
Bottom data acquisition is realized by OPC UA, and the PLC label point acquired will be needed to be configured to the address OPC UA browsing
Space file, selects PLC type number, configuration of IP address port number, and starting OPC UA server and PLC establish TCP connection.Pass through
OPC UA client can collect PLC label point data.Development interface server is by the OPC UA normal data lattice after parsing
Formula is automatically converted to the Key-Value format of Redis storage.
PLC controller is counted according to different label vertex types, such as type of alarm, technological parameter type, yield, distribution
It is responsible for monitoring to different Redis nodes, different application connects different nodes and adopts to required PLC label point progress data
Collection.In this way memory source, while different application programs can be rationally utilized with the quantity of the slot of each monitoring nodes of flexible configuration
Between be independent of each other, realize different business module stand-alone development.
AOF mode is set by Redis persistence, under AOF mode, each executed can be updated life by Redis
Order is recorded, and is saved in AOF file；When Redis node needs to restore database data, it is only necessary to from previously stored
Order, execution are successively read in AOF file.To prevent AOF file excessive, it can be set and be written over AOF file, i.e.,
Merge to the multiple write operations of same data, and by the AOF file backup of newest rewriting to different from node.
When there is host node to break down, other host nodes can monitor exception, for example, can be by constantly sending
Whether Ping command determination survives, and one of them is promoted to host node from node.Meanwhile the client of connecting fault host node
The order that end can receive redirection reconnects to new host node according to the IP address and port numbers in order.
Through the above steps, the real-time read-write operation of mass data may be implemented, solution is led because of local server delay machine
The data loss problem and upper layer application no data of cause carry out source problem.
According to embodiments of the present invention, a kind of embodiment of the method for acquiring data is provided, Fig. 3 is according to embodiments of the present invention
Another acquisition data method flow chart, as shown in figure 3, this method comprises the following steps:
Step S302, the host node in device clusters issue acquisition request, wherein device clusters include at least: host node
It is the backup of host node from node at least one from node；
Step S304 is based on relational data structure, obtains acquisition and requests requested target data, wherein relation data
Structure includes at least: the corresponding relationship between host node and communication intermediate station communicates corresponding between intermediate station and target data
Relationship.
Wherein, communication intermediate station is for converting target data format.Intermediate station is communicated by the format of target data
Be converted to Key-Value format.
In the embodiment of the present application, each node in cluster and other nodes establish communication connection, which can
To be TCP communication.Host node is used for the acquisition and storage to data, from the node for the duplication that node is host node.Pass through node
Between establish TCP communication and can achieve host node other host nodes are monitored, host node detects whether other host nodes are sent out
Any one is promoted to host node from node in the case where detecting that host node breaks down by raw failure.Wherein, main section
Point can send Ping command determination to other nodes, and whether host node breaks down.
In the case where host node breaks down, any one in cluster is promoted to host node from node.Connecting fault
The communication intermediate station (or OPC OA client) of host node receive redirect instruction after, according in instruction IP address and
Port numbers, communication intermediate station are connected to new host node, redirect instruction for that will communicate intermediate station and are connected to new host node.
In the embodiment of the present application, write instruction is saved in AOF file by host node interval preset time.Host node will
The write instruction that same data carry out is merged in AOF file, obtains merging file；Host node will merge file backup at least
One from node.
According to embodiments of the present invention, a kind of Installation practice for acquiring data is provided, Fig. 4 is according to embodiments of the present invention
A kind of acquisition data device structure chart, as shown in figure 4, the device includes:
Transmission unit 40 issues acquisition request for the host node in device clusters, wherein device clusters include at least:
Host node, from node, is the backup of host node from node at least one；
Acquiring unit 42 obtains acquisition and requests requested target data, wherein close for being based on relational data structure
Be that data structure includes at least: the corresponding relationship between host node and communication intermediate station communicates between intermediate station and target data
Corresponding relationship.Wherein, communication intermediate station is for converting target data format.
In the embodiment of the present application, data set is acquired further include: storage unit, it will for host node interval preset time
Write instruction is saved in AOF file.
Combining unit will merge the write instruction that same data carry out in AOF file for host node, obtain merging text
Part；Host node is by merging file backup at least one from node.
According to embodiments of the present invention, a kind of Installation practice for acquiring data is provided, Fig. 5 is according to embodiments of the present invention
Another acquisition data device structure chart, as shown in figure 5, the device includes:
Acquiring unit 50, for obtaining target data corresponding device and communicating the first corresponding relationship between intermediate station, and
Obtain the second corresponding relationship between the host node in communication intermediate station and cluster；
Acquisition unit 52, if for getting the corresponding relationship between host node and target data corresponding device, main section
Point acquisition target data, wherein include at least host node in cluster at least one from node, be the standby of host node from node
Part.
In the embodiment of the present application, data set is acquired further include: connection unit, for being built according to IP address and port numbers
Corresponding relationship between vertical host node and communication intermediate station, communication intermediate station and target data corresponding device, wherein communication is intermediate
It stands for being converted to target data format.
Connection unit includes: conversion module, for the format of target data to be converted to key-value pair format.
Unit is redirected, for detecting whether host node breaks down；In the case where host node breaks down, by cluster
In any one from node be promoted to host node；After receiving and redirecting instruction, according to the IP address and port numbers in instruction,
Communication intermediate station is connected to new host node, instruction is redirected for intermediate station will to be communicated and is connected to new host node.
Write instruction is saved in AOF file by storage unit for being spaced preset time.
Combining unit obtains merging file for will merge the write instruction that same data carry out in AOF file；It will
Merging file backup is at least one from node.
Recovery unit, for when the data of node need to restore, executing all instructions that AOF file saves.
According to embodiments of the present invention, a kind of storage medium is provided, storage medium includes the program of storage, wherein in journey
The method that equipment where controlling storage medium when sort run executes above-mentioned acquisition data.
According to embodiments of the present invention, a kind of processor is provided, processor is for running program, wherein when program is run
The method for executing above-mentioned acquisition data.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
In the above embodiment of the invention, it all emphasizes particularly on different fields to the description of each embodiment, does not have in some embodiment
The part of detailed description, reference can be made to the related descriptions of other embodiments.
In several embodiments provided herein, it should be understood that disclosed technology contents can pass through others
Mode is realized.Wherein, the apparatus embodiments described above are merely exemplary, such as the division of the unit, Ke Yiwei
A kind of logical function partition, there may be another division manner in actual implementation, for example, multiple units or components can combine or
Person is desirably integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutual
Between coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or communication link of unit or module
It connects, can be electrical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
On unit.It can some or all of the units may be selected to achieve the purpose of the solution of this embodiment according to the actual needs.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially
The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words
It embodies, which is stored in a storage medium, including some instructions are used so that a computer
Equipment (can for personal computer, server or network equipment etc.) execute each embodiment the method for the present invention whole or
Part steps.And storage medium above-mentioned includes: that USB flash disk, read-only memory (ROM, Read-OnlyMemory), arbitrary access are deposited
Reservoir (RAM, Random Access Memory), mobile hard disk, magnetic or disk etc. be various to can store program code
Medium.
The above is only a preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also answered
It is considered as protection scope of the present invention.
Claims (15)
1. a kind of method for acquiring data characterized by comprising
Host node in device clusters issues acquisition request, wherein the device clusters include at least: the host node and at least
One from node, described is the backup of the host node from node；
Based on relational data structure, obtains the acquisition and request requested target data, wherein the relational data structure is extremely
It less include: the corresponding relationship between the host node and communication intermediate station, between the communication intermediate station and the target data
Corresponding relationship.
2. the method according to claim 1, wherein the communication intermediate station is used to carry out target data format
Conversion.
3. the method according to claim 1, wherein the host node is acquired it to the target data
Afterwards, comprising:
Write instruction is saved in AOF file by host node interval preset time.
4. according to the method described in claim 3, it is characterized in that, write instruction is saved in by host node interval preset time
After in AOF file, comprising:
Host node will merge the write instruction that same data carry out in the AOF file, obtain merging file；
Host node is by the merging file backup at least one from node.
5. a kind of method for acquiring data characterized by comprising
Obtain target data corresponding device and communicate intermediate station between the first corresponding relationship, and obtain the communication intermediate station and
The second corresponding relationship between host node in cluster；
If getting the corresponding relationship between the host node and the target data corresponding device, the host node acquires institute
State target data, wherein included at least in the cluster host node and at least one from node, it is described from node be the main section
The backup of point.
6. according to the method described in claim 5, it is characterized in that, before the host node acquisition target data, comprising:
Host node and communication intermediate station, communication intermediate station and the target data corresponding device are established according to IP address and port numbers
Between corresponding relationship, wherein the communication intermediate station is for converting target data format.
7. according to the method described in claim 6, it is characterized in that, the communication intermediate station is used to carry out target data format
Conversion, comprising:
The format of the target data is converted into key-value pair format.
8. according to the method described in claim 5, it is characterized in that, after the host node acquisition target data, comprising:
Detect whether the host node breaks down；
In the case where the host node breaks down, any one in cluster is promoted to host node from node；
After receiving redirection instruction, according to the IP address and port numbers redirected in instruction, among the communication
Station is connected to new host node, and the redirection instruction is for being connected to new host node for the communication intermediate station.
9. according to the method described in claim 5, it is characterized in that, after the host node acquisition target data, comprising:
Write instruction is saved in AOF file by interval preset time.
10. according to the method described in claim 9, it is characterized in that, write instruction is saved in AOF file by interval preset time
In after, comprising:
The write instruction that same data carry out will be merged in the AOF file, obtains merging file；
By the merging file backup at least one from node.
11. according to the method described in claim 9, it is characterized in that, write operation is saved in AOF file by interval preset time
In after, comprising:
When the data of node need to restore, all instructions that the AOF file saves is executed.
12. a kind of device for acquiring data characterized by comprising
Transmission unit issues acquisition request for the host node in device clusters, wherein the device clusters include at least: institute
State host node and at least one from node, described is the backup of the host node from node；
Acquiring unit obtains the acquisition and requests requested target data, wherein is described for being based on relational data structure
Relational data structure includes at least: the corresponding relationship between the host node and communication intermediate station, the communication intermediate station and institute
State the corresponding relationship between target data.
13. a kind of device for acquiring data characterized by comprising
Acquiring unit for obtaining target data corresponding device and communicating the first corresponding relationship between intermediate station, and obtains institute
State the second corresponding relationship between the host node in communication intermediate station and cluster；
Acquisition unit, if for getting the corresponding relationship between the host node and the target data corresponding device, institute
State host node and acquire the target data, wherein included at least in the cluster host node and at least one from node, it is described from
Node is the backup of the host node.
14. a kind of storage medium, which is characterized in that the storage medium includes the program of storage, wherein run in described program
When control the storage medium where equipment perform claim require any one of 1 to 4 or 5 to 11 described in acquisition data side
Method.
15. a kind of processor, which is characterized in that the processor is for running program, wherein right of execution when described program is run
Benefit require any one of 1 to 4 or 5 to 11 described in acquisition data method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811190166.9A CN109284294A (en) | 2018-10-12 | 2018-10-12 | Acquire method and device, the storage medium, processor of data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811190166.9A CN109284294A (en) | 2018-10-12 | 2018-10-12 | Acquire method and device, the storage medium, processor of data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109284294A true CN109284294A (en) | 2019-01-29 |
Family
ID=65177548
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811190166.9A Pending CN109284294A (en) | 2018-10-12 | 2018-10-12 | Acquire method and device, the storage medium, processor of data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109284294A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110910177A (en) * | 2019-11-28 | 2020-03-24 | 中国银行股份有限公司 | Data acquisition system |
| CN112269788A (en) * | 2020-11-13 | 2021-01-26 | 中盈优创资讯科技有限公司 | Method and device for improving click House data storage performance |
| CN112416847A (en) * | 2020-04-15 | 2021-02-26 | 上海哔哩哔哩科技有限公司 | Redis cluster creating method, system and device |
| CN115665020A (en) * | 2022-11-10 | 2023-01-31 | 广州市易鸿智能装备有限公司 | Communication analysis method, device, equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080209007A1 (en) * | 2007-02-27 | 2008-08-28 | Tekelec | Methods, systems, and computer program products for accessing data associated with a plurality of similarly structured distributed databases |
| CN104133774A (en) * | 2013-05-02 | 2014-11-05 | 擎泰科技股份有限公司 | Method of managing non-volatile memory and non-volatile storage device thereof |
| CN104656572A (en) * | 2013-11-20 | 2015-05-27 | 田锐 | Internet of Things household control system based on handheld terminal |
| CN106773766A (en) * | 2016-12-31 | 2017-05-31 | 广东博意建筑设计院有限公司 | Smart home house keeper central control system and its control method with learning functionality |
| CN107193909A (en) * | 2017-05-08 | 2017-09-22 | 杭州东方通信软件技术有限公司 | Data processing method and system |
| CN107861859A (en) * | 2017-11-22 | 2018-03-30 | 北京汇通金财信息科技有限公司 | A kind of blog management method and system based on micro services framework |
| CN108052086A (en) * | 2017-12-28 | 2018-05-18 | 珠海格力智能装备技术研究院有限公司 | Data collecting system |
-
2018
- 2018-10-12 CN CN201811190166.9A patent/CN109284294A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080209007A1 (en) * | 2007-02-27 | 2008-08-28 | Tekelec | Methods, systems, and computer program products for accessing data associated with a plurality of similarly structured distributed databases |
| CN104133774A (en) * | 2013-05-02 | 2014-11-05 | 擎泰科技股份有限公司 | Method of managing non-volatile memory and non-volatile storage device thereof |
| CN104656572A (en) * | 2013-11-20 | 2015-05-27 | 田锐 | Internet of Things household control system based on handheld terminal |
| CN106773766A (en) * | 2016-12-31 | 2017-05-31 | 广东博意建筑设计院有限公司 | Smart home house keeper central control system and its control method with learning functionality |
| CN107193909A (en) * | 2017-05-08 | 2017-09-22 | 杭州东方通信软件技术有限公司 | Data processing method and system |
| CN107861859A (en) * | 2017-11-22 | 2018-03-30 | 北京汇通金财信息科技有限公司 | A kind of blog management method and system based on micro services framework |
| CN108052086A (en) * | 2017-12-28 | 2018-05-18 | 珠海格力智能装备技术研究院有限公司 | Data collecting system |
Non-Patent Citations (2)
| Title |
|---|
| 付磊，张益军著: ""Redis开发与运维"", 《REDIS开发与运维》 * |
| 黄英波等: ""利用CDC/Redis等技术提升实时流量营销支撑能力"", 《电信技术》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110910177A (en) * | 2019-11-28 | 2020-03-24 | 中国银行股份有限公司 | Data acquisition system |
| CN112416847A (en) * | 2020-04-15 | 2021-02-26 | 上海哔哩哔哩科技有限公司 | Redis cluster creating method, system and device |
| CN112416847B (en) * | 2020-04-15 | 2023-05-02 | 上海哔哩哔哩科技有限公司 | Redis cluster creation method, system and device |
| CN112269788A (en) * | 2020-11-13 | 2021-01-26 | 中盈优创资讯科技有限公司 | Method and device for improving click House data storage performance |
| CN115665020A (en) * | 2022-11-10 | 2023-01-31 | 广州市易鸿智能装备有限公司 | Communication analysis method, device, equipment and storage medium |
| CN115665020B (en) * | 2022-11-10 | 2023-04-28 | 广州市易鸿智能装备有限公司 | Communication analysis method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109284294A (en) | Acquire method and device, the storage medium, processor of data | |
| CN107733986B (en) | Protection operation big data supporting platform supporting integrated deployment and monitoring | |
| US20210081432A1 (en) | Configurable data replication | |
| US11360689B1 (en) | Cloning a tracking copy of replica data | |
| US20220263898A1 (en) | Supporting Multiple Replication Schemes Across Distinct Network Layers | |
| CN104965850B (en) | A kind of database high availability implementation method based on open source technology | |
| CN106713487A (en) | Data synchronization method and device | |
| US11789638B2 (en) | Continuing replication during storage system transportation | |
| CN109218100A (en) | Distributed objects storage cluster and its request responding method, system and storage medium | |
| JP2017529590A (en) | Centralized analysis of application, virtualization and cloud infrastructure resources using graph theory | |
| KR20150112357A (en) | Sensor data processing system and method thereof | |
| EP2606607B1 (en) | Determining equivalent subsets of agents to gather information for a fabric | |
| CN102088490B (en) | Data storage method, device and system | |
| CN104506348A (en) | Method for automatically discovering and configuring monitoring object | |
| CN104252500A (en) | Method and device for carrying out fault repairing on database management platform | |
| CN106528341B (en) | Automation disaster tolerance system based on Greenplum database | |
| CN103546313A (en) | Cloud computing based IT (information technology) operation and maintenance management system | |
| CN103488793A (en) | User behavior monitoring method based on information retrieval | |
| CN105580007A (en) | Monitoring a computing network | |
| CN110807064A (en) | Data recovery device in RAC distributed database cluster system | |
| CN107104820B (en) | Dynamic capacity-expansion daily operation and maintenance method based on F5 server node | |
| CN102638378A (en) | Mass storage system monitoring method integrating heterogeneous storage devices | |
| CN113176948A (en) | Edge gateway, edge computing system and configuration method thereof | |
| CN103973516A (en) | Method and device for achieving monitoring function in data processing system | |
| US20210263931A1 (en) | Elimination of Measurement Lag for Operations Across a Large Number of Customer Nodes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190129 |