CN107065800B - Industrial signal data access method based on fixed length block - Google Patents
Industrial signal data access method based on fixed length block Download PDFInfo
- Publication number
- CN107065800B CN107065800B CN201710285624.6A CN201710285624A CN107065800B CN 107065800 B CN107065800 B CN 107065800B CN 201710285624 A CN201710285624 A CN 201710285624A CN 107065800 B CN107065800 B CN 107065800B
- Authority
- CN
- China
- Prior art keywords
- data
- measuring point
- server
- sig
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Computer And Data Communications (AREA)
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Manufacturing & Machinery (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to the industrial signal data access method based on fixed length block, solves the low defect for causing data to extract the wasting of resources of data user rate that industrial signal source is extracted compared with prior art.The present invention is the following steps are included: building inner server cluster；Measuring point is encoded on inner server；Receive data transmitted by data high-speed bus；The reception and processing of inner server data；Request server or client read data.Present invention reduces the calculated load of server retrieval data, the technology key data for meeting client show in real time and the real-time of industrial curve report shows requirement.
Description
Technical field
The present invention relates to industrial data access technique fields, specifically the industrial signal data based on fixed length block
Access method.
Background technique
It is perfect with the business such as Internet of Things, cloud computing, big data and technology, and national 2025 strategy of made in China pushes away
Dynamic, industrial data center is collected and is stored, Distribution Center as industrial data, will play irreplaceable role.In work
In the everyday scenes of industry application, need industrial data center that can efficiently collect discrete industrial measuring point data signal, and
Data can be retrieved, efficiently to meet the requirement of industry spot Real-time process data exhibiting and real-time curve report.And it is
Meet the high speed access of industrial implementation signal, it is using efficient data buffer storage administrative skill essential.
In industrial circle, there are many storage of live signal and retrieval scheme, and wherein typical scenario is typically via reality
When database especially RTDB in Industry Control come realize measuring point data cache.There is real-time data base high healthy and strong, height to handle up, thing
Be engaged in the features such as safe, but since it is expensive, be often used in MES, MRP system of large enterprise, for data center this
Its cost is unacceptable for the demand that sample is provided infrastructures for conglomerate and is not necessarily to.
Also there are many technical solutions to use and distributed memory data queue is built to solve work based on Memchched, Redis etc.
Industry data buffer storage problem, it is inadequate for carrying out storing data only with K-Value mode, it is still necessary to build distributed data cache shelf
Structure simultaneously builds corresponding data structure according to business actual needs and defines access strategy.Need it is to be noted that
The distributed variable-frequencypump frame Spark memory-based of UCBerkeley and the Storm based on MapReduce are provided which
Data processing mechanism based on stream process solves the real-time computational problem of big data, but slow as simple data
It rushes and improper.
Memory computing technique in research field, big data Technology Chain is gradually paid close attention to by people in recent years, country in 2015
" the memory key technology towards big data " is classified as the important content of information technology field by 863 Program.But domestic scientific research
The scientific achievement of mechanism relatively emphasizes learning value, and there are certain difficulties for achievements conversion.With the manufacture of computer hardware equipment
The promotion of technology and super integration technology it is universal, using super integration technology framework or inexpensive vast capacity memory service
Device can bring new solution for the industrial data caching at industrial data center.
For the angle at industrial data center, industrial data center generally requires to converge a large amount of signal data.Assuming that
One group of industrial website of server access more than 1000, each industry website need to monitor 1000 signal measuring points, and each measuring point needs
2000 datas of retrieval are saved in real time, then server needs to provide preservation and retrieval service for nearly 200,000,000 records.Even if these
Data all use memory to save, and if not being considered as suitable Data Structure and Algorithm, then it is huge to calculate pressure for server retrieval
Greatly, can not summary responses and processing high frequency time request of data.From construction cost angle, data center also should not be only
Meet data buffering to need and use expensive very large memory server.
From the point of view of the practical application that industry remotely monitors, long-range monitoring generally requires consideration observation process node key index
Data, and the real-time curve of a certain key index time zone of observation.And the instant computing of live signal and in real time letter
Number backtracking often industrial environment scene and local information center processing.Therefore be analyzed as follows: long-range monitoring is to technique
It when node monitors, needs to observe measuring point latest data referring to process structure figure, this is that it is only necessary to servers sends to client
The measuring point single latest data；And when observing measuring point real-time curve, it is limited to display device resolution size, clothes
Being engaged in, it is only necessary to the measuring point data of regular length is sent to client for device.
Therefore, the mass data extracted in traditional technology is practical to be used in the then considerably less of client, causes data
The wasting of resources of extraction process.How using long-range this data extraction property monitored in practical applications of industry, work is reduced
The extracted amount of industry source point data designs a kind of industrial signal data access method and has become technical problem urgently to be solved.
Summary of the invention
Data are caused the purpose of the present invention is to solve the data user rate that industrial signal source is extracted in the prior art is low
The defect for extracting the wasting of resources provides a kind of industrial signal data access method based on fixed length block to solve above-mentioned ask
Topic.
To achieve the goals above, technical scheme is as follows:
A kind of industrial signal data access method based on fixed length block, comprising the following steps:
Inner server cluster is constructed, needs to set data block length according to the generation of measuring point real-time curve, memory is taken
Business device carries out memory configurations, and establishes inner server informational table of nodes；
Measuring point is encoded on inner server, the measuring point stored respectively to it on every inner server carries out
Encode and establish the Memory Allocation structure of inner server；
Data transmitted by data high-speed bus are received, data are written scheduler and receive number transmitted by data high-speed bus
According to, and data are stored in inner server；
The reception and processing of inner server data, inner server receive the data of data write-in scheduler dispatches, and
It is handled, updates corresponding data block；
Request server or client read data, and client reads corresponding data, with generate measuring point real-time curve or
Show Real-time process data.
The building inner server cluster the following steps are included:
The demand that measuring point real-time curve is generated according to client, set data block length as 128,256,512,1024 or
2048；
Mark sig_id is established to measuring point, mark is numbered from 0；
Calculate every inner server memory size, according to data block length distribute every inner server corresponding to survey
Points, and corresponding measuring point is assigned to every inner server；
Inner server informational table of nodes is established, establishes array sig_id_ in memory server node information table
Array, array index sig_id, array sig_id_array store signal name and measuring point identity map, and signal name is
The corresponding measuring point of sig_name, sig_name is identified as sig_id；
It establishes and saves the corresponding inner server information server_element of measuring point data access, inner server information
Server_element structure is as follows:
{min_sig_id,max_sig_id,server_id,server_address},
Wherein: min_sig_id is the minimum measuring point number of inner server storage, and max_sig_id is memory clothes
The maximum measuring point number of business device storage, serve_id are inner server number, and server_address is inner server visit
Ask address；
Several server_element form inner server measuring point allocation table sig_server_list；
Inner server measuring point allocation table sig_server_list is saved in memory server node information table.
It is described measuring point is encoded on inner server the following steps are included:
Configuration indexes in each inner server, and it includes single measuring point datas to index mono_ind_array, number
Block_ind_array is indexed according to block；
Single measuring point data index mono_ind_array be address one-dimension array, array index id since 0, wherein
The corresponding relationship of subscript id and sig_id is
Id=sig_id-min_sig_id；
Data block index block_ind_array be two-dimensional array, wherein each element structure be pos, length,
address},
Pos is used to indicate current newest measuring point data storage position, and length is used to indicate entire data block length,
Address is used to indicate data block memory first address；
Queue is configured in each inner server, it includes the queue of single measuring point data and measuring point data block queue,
Wherein the queue of single measuring point data is used for for storing the last received measuring point data, measuring point data block queue according to client
End requires to store the measuring point data in nearest a period of time, and the data block length fixed size of setting is pressed in the distribution of measuring point data block,
It is stored using independent memory headroom；
All inner server data are initialized, corresponding memory address is filled to measuring point index, successively to
The first address for the memory address that mono_ind_array, block_ind_array filling distribute.
Data transmitted by the reception data high-speed bus the following steps are included:
Data be written scheduler from high speed data bus receive data, data mode be sig_name, timestamp,
Value }, sig_name is the hashed value of signal name, and timestamp is time tag, and value is measuring point signal value measuring point letter
Number numerical value got；
Data are written scheduler and use binary chop, and sig_name is searched from sig_id_array, obtains sig_
The array index of name is sig_id；Sig_server_list, which is retrieved, according to sig_id finds inner server address；
Data write-in scheduler sends the data to the inner server found.
The receptions of the inner server data and processing the following steps are included:
Inner server receives data, is saved in single measuring point data；
It calculates and is designated as i_cursor=sig_id-min_sig_id under single measuring point data index and obtains measuring point to take currently
The single-point index position of business device；The memory that measuring point data is stored is obtained at index mono_ind_array [i_cursor]
Data are written in this address in location；
Inner server receives data, saves data to measuring point data block, steps are as follows:
Calculate i_cursor=sig_id-min_sig_id；Measuring point is obtained to index in the measuring point data block of current server
Position；
Measuring point data storage data block first address block_ is obtained from block_ind_array [i_cursor]
Address, and obtain the current measuring point data storage position pos of data block, data block length len；
Calculate pos=pos mod len；In block_address [pos] storage measuring point signal data record；Current
Position covers newest measuring point signal data；
Block_ind_array [i_cursor] current location pos is updated to pos+1mod len.
The request server or client read data packet includes following steps:
Client transmit a request to data access scheduler, request comprising sig_name and request data be single also
It is data block,
According to the signal name sig_name of request, measuring point mark sig_id is found from sig_id_array；It checks
Sig_server_list searches corresponding inner server, transmits the request to corresponding inner server；
If request is reads single measuring point data, inner server is from mono_ind_array [sig_id-min_sig_
Id] at directly read data record and place extra buffer；
If request is signal real-time curve, inner server is from block_address=
Data block full content is directly read at block_ind_array [sig_id-min_sig_id] places interim delay
Rush area；
Data block contents are arranged in extra buffer, steps are as follows:
The data of block_address [0..pos-1] and block_address [pos..len-1] data content are handed over
Extra buffer is put into after changing；
Encoder according to request server or client requirements by extra buffer data encoding at XML format or
JSON format, places output queue, and output queue is output to request server or client using FIFO mode.
Beneficial effect
Industrial signal data access method based on fixed length block of the invention, it is remote using industry compared with prior art
The data characteristic of range monitoring uses the queue of single measuring point and data block queue storing data, and improves memory using indexed mode
Retrieval rate of the server to industrial signal.The present invention is all taken out the measuring point data of certain time sequence using indexed mode
And send, to reduce service retrieval load, to reduce the calculated load of server retrieval data, meet the work of client
Skill critical data shows in real time and the real-time of industrial curve report shows requirement.
Detailed description of the invention
Fig. 1 is method precedence diagram of the invention；
Fig. 2 is the memory allocation map of inner server in the present invention；
Fig. 3 is that industrial signal data access connects block diagram in the prior art.
Specific embodiment
The effect of to make to structure feature of the invention and being reached, has a better understanding and awareness, to preferable
Examples and drawings cooperation detailed description, is described as follows:
As shown in figure 3, high speed data bus is used for memory in the industrial signal data access connection relationship of the prior art
Real-time measuring point data is written in server group, can use ZMQ, RabbitMQ, ZeroMQ, JMS, MSMQ, ActiveMQ, Jafka/
Any one in Kafka is realized.
Inner server informational table of nodes is for saving inner server measuring point allocation table sig_server_list and array
sig_id_array。
Data are written scheduler and read measuring point signal data from high speed data bus, and inner server is selected to save data.
Inner server is for storing and retrieve data, and Memory Allocation graph structure is as shown in Fig. 2, wherein single measuring point
Data directory saves measuring point data block first address and data for storing every measuring point data address, measuring point data block index
Block length, when presubscript information.Single measuring point data queue saves each measuring point data record, and measuring point data record format is
{ sig_name, time_stamp, value }.Measuring point data block saves the measuring point data record in certain time sequence, measuring point number
It is regular length according to block.
Data access scheduler receives request server or client data request, selects the existing service of measuring point signal
Device.What it is such as request is single technique measuring point signal data, then takes out single measuring point data record from correspondence memory server.Such as ask
What is asked is one section of real-time curve data, then all data records of entire data block are taken out from correspondence memory server, collated
After provide.
The data that encoder takes out data access scheduling device are encoded by request server or client requirements.
Output caching waits request server or client to consume number for the measuring point data record after storing coding
According to.
As shown in Figure 1, the industrial signal data access method of the present invention based on fixed length block, including following step
It is rapid:
The first step constructs inner server cluster.Need to set data block length according to the generation of measuring point real-time curve, it is right
Inner server carries out memory configurations, and establishes inner server informational table of nodes.Here, to reduce the time-consuming of data query,
Corresponding inner server is distributed for each industrial data node, and corresponding index is established to inner server.Carry out
When industrial node data is inquired, without inquiring traditional database or accessing industrial node, corresponding internal storage data need to be only accessed
Library.The specific steps of which are as follows:
(1) demand that measuring point real-time curve is generated according to client, sets data block length as 128,256,512,1024
Or 2048, data block length is as unit of data bit.
(2) mark sig_id is established to measuring point (industrial signal node), mark is numbered from 0.
(3) calculate every inner server memory size, distribute every inner server according to data block length corresponding to
Measuring point number, and corresponding measuring point is assigned to every inner server.Such as: it amounts to and is equipped with 4 inner servers, in every
It deposits server and can provide 32G memory for storing data block, if measuring point data block length is 1K, then every server can divide
It is 32M with measuring point number.
(4) inner server informational table of nodes is established.Array sig_id_ is established in memory server node information table
Array, array are designated as sig_id under being, array sig_id_array stores signal name and measuring point identity map, signal name
For sig_name, the corresponding measuring point of sig_name is identified as sig_id.To use hash-coding convenient for quick-searching sig_name.
(5) it establishes and saves the corresponding inner server information server_element of measuring point data access, inner server
Information server_element structure is as follows:
{min_sig_id,max_sig_id,server_id,server_address},
Wherein: min_sig_id is the minimum measuring point number of inner server storage, and max_sig_id is memory clothes
The maximum measuring point number of business device storage, serve_id are inner server number, and server_address is inner server visit
Ask address.
Several server_element form inner server measuring point allocation table sig_server_list.
(6) inner server measuring point allocation table sig_server_list is saved in memory server node information table.
Second step encodes measuring point on inner server.It is stored respectively on every inner server
Measuring point is encoded and establishes the Memory Allocation structure of inner server.During for access inner server, also simultaneously
It is unconventional to be searched for inner server, but take and directly transfer (non-lookup) from corresponding inner server
Method.For in actual use, needing the data for observing measuring point latest data referring to process structure figure to transfer mode, only need to read
Single measuring point data queue, and in single measuring point data queue it is the number for the single-point (a certain industry signal source) constantly updated
According to, then be equipped with single measuring point data index and positioned；And when need to generate measuring point real-time curve, then it needs to read measuring point data block team
Column, saving in measuring point data block queue is data in single-point (a certain industry signal source) data block length, then is equipped with data
Block index is positioned, to reduce calculating and the response time of system, improves operational efficiency.The specific steps of which are as follows:
(1) in each inner server configuration index, it includes single measuring point data index mono_ind_array,
Data block indexes block_ind_array, and single measuring point data index mono_ind_array is the one-dimension array of an address.
Array index id is since 0.The wherein corresponding relationship of subscript id and sig_id are as follows: id=sig_id-min_sig_id；Data block
Indexing block_ind_array is a two-dimensional array, wherein each element structure is { pos, length, address }, pos
It is used to indicate current newest measuring point data storage position, length is used to indicate entire data block length, and address is for referring to
Show data block memory first address.
Queue is configured in each inner server, it includes the queue of single measuring point data and measuring point data block queue,
Wherein the queue of single measuring point data is used for for storing the last received measuring point data, measuring point data block queue according to client
End requires to store the measuring point data in nearest a period of time, and the data block length fixed size of setting is pressed in the distribution of measuring point data block,
It is stored using independent memory headroom.
(2) all inner server data are initialized, corresponding memory address is filled to measuring point index.Specifically
Successively to the first address of mono_ind_array, block_ind_array the filling memory address distributed.
Third step receives data transmitted by data high-speed bus.Data are written scheduler and receive data high-speed bus institute
The data of transmission, and data are stored in inner server.The specific steps of which are as follows:
(1) data be written scheduler from high speed data bus receive data, data mode be sig_name,
Timestamp, value }, in which: sig_name is the hashed value of signal name, and timestamp is time tag, and value is
The numerical value that measuring point signal value measuring point signal is got.
(2) data write-in scheduler uses binary chop, sig_name is searched from sig_id_array, by the first step
(4) it is found that the array index of corresponding sig_name is sig_id；It is found according to sig_id retrieval sig_server_list interior
Deposit server address.
(3) data write-in scheduler sends the data to the inner server found.
It can thus be seen that the present invention is that the lookup of inner server is carried out in data write-in scheduler, i.e., it is traditional
Industrial signal big data source lookup, become herein carry out inner server lookup, find out required memory service
Device address greatly reduces data search amount.
4th step, the reception and processing of inner server data.Inner server receives data write-in scheduler dispatches
Data, and handled, update corresponding data block.In view of each measuring point data may be used as technology key index or
It is as process curve a part, it is therefore desirable to save simultaneously, the specific steps are as follows:
(1) inner server receives data, is saved in single measuring point data；
It calculates and is designated as i_cursor=sig_id-min_sig_id under single measuring point data index and obtains measuring point to take currently
The single-point index position of business device；The memory that measuring point data is stored is obtained at index mono_ind_array [i_cursor]
Data are written in this address in location；
(2) inner server receives data, saves data to measuring point data block, steps are as follows:
Calculate i_cursor=sig_id-min_sig_id；Measuring point is obtained to index in the measuring point data block of current server
Position；
Measuring point data storage data block first address block_ is obtained from block_ind_array [i_cursor]
Address, and obtain the current measuring point data storage position pos of data block, data block length len；
Calculate pos=pos mod len；In block_address [pos] storage measuring point signal data record；Current
Position covers newest measuring point signal data.
Block_ind_array [i_cursor] current location pos is updated to pos+1mod len.It is used to indicate next
Measuring point data storage position.
Here, inner server constantly carries out the update of data, to wait request server or client to carry out data reading
It takes.
Thus mode can constantly refresh in the data block memory of fixed size, save the survey in nearest a period of time
Point signal data.
5th step, request server or client read data.Client reads corresponding data, bent in real time to generate measuring point
Line shows Real-time process data.The specific steps of which are as follows:
(1) client transmit a request to data access scheduler, and request is single comprising sig_name and request data
Or data block.
According to the signal name sig_name of request, measuring point mark sig_id is found from sig_id_array；It checks
Sig_server_list searches corresponding inner server, transmits the request to corresponding inner server.
(2) if request is reads single measuring point data, inner server is from mono_ind_array [sig_id-min_
Sig_id] at directly read data record and place extra buffer.
(3) if request is signal real-time curve, inner server is from block_address=block_ind_array
Data block full content is directly read at [sig_id-min_sig_id] places extra buffer；
(4) data block contents are arranged in extra buffer,
Steps are as follows:
The data of block_address [0..pos-1] and block_address [pos..len-1] data content are handed over
It is put into extra buffer after changing, guarantees that data block contents are to be sequentially arranged；
(5) encoder according to request server or client requirements by extra buffer data encoding at XML format or
Person's JSON format, places output queue, and output queue is output to request server or client using FIFO mode.
In industry spot monitoring system, it is desirable to be able to show in real time live measuring point data critical process index or
Show the real-time curve in nearest a period of time.And the preservation that the industrial data center serviced for it is required to high speed is a large amount of
Real-time live signal data simultaneously being capable of quickly interviewee's retrieval.The present invention uses inner server measuring point allocation table and measuring point
Index the different inner servers and memory address of two-level configuration instruction unlike signal data access.The present invention uses single
Measuring point queue and data block queue save signal data, and single measuring point queue is for storing newest one-time process node key index
Data, data block queue save the process node measuring point signal data in nearest a period of time, due to using directly covering side
The memory mechanism of formula, therefore do not save historical data.Using the access mode of mechanism as above, visitor is made directly memory to be utilized to take
Business device measuring point allocation table and measuring point index obtain data, avoid the retrieval of convential memory data, greatly reduce memory service
The retrieval pressure of device is suitble to industrial data center using the field of a large amount of industrial signal datas of distributed memory server centered processing
Scape.
The basic principles, main features and advantages of the present invention have been shown and described above.The technology of the industry
Personnel are it should be appreciated that the present invention is not limited to the above embodiments, and what is described in the above embodiment and the description is only the present invention
Principle, various changes and improvements may be made to the invention without departing from the spirit and scope of the present invention, these variation and
Improvement is both fallen in the range of claimed invention.The present invention claims protection scope by appended claims and its
Equivalent defines.
Claims (6)
1. a kind of industrial signal data access method based on fixed length block, which comprises the following steps:
11) inner server cluster is constructed, needs to set data block length according to the generation of measuring point real-time curve, to memory service
Device carries out memory configurations, and establishes inner server informational table of nodes；
12) measuring point is encoded on inner server, the measuring point stored respectively to it on every inner server carries out
Encode and establish the Memory Allocation structure of inner server；
13) data transmitted by data high-speed bus are received, data are written scheduler and receive number transmitted by data high-speed bus
According to, and data are stored in inner server；
14) reception and processing of inner server data, inner server receive the data of data write-in scheduler dispatches, go forward side by side
Row processing, updates corresponding data block；
15) request server or client read data, and client reads corresponding data, to generate measuring point real-time curve or exhibition
Show Real-time process data.
2. the industrial signal data access method according to claim 1 based on fixed length block, which is characterized in that described
Building inner server cluster the following steps are included:
21) demand that measuring point real-time curve is generated according to client, set data block length as 128,256,512,1024 or
2048；
22) mark sig_id is established to measuring point, mark is numbered from 0；
23) calculate every inner server memory size, according to data block length distribute every inner server corresponding to survey
Points, and corresponding measuring point is assigned to every inner server；
24) establish inner server informational table of nodes, established in memory server node information table array sig_id_array,
Array index is sig_id, and array sig_id_array stores signal name and measuring point identity map, signal name sig_
The corresponding measuring point of name, sig_name is identified as sig_id；
25) it establishes and saves the corresponding inner server information server_element of measuring point data access, inner server information
Server_element structure is as follows:
{min_sig_id,max_sig_id,server_id,server_address},
Wherein: min_sig_id is the minimum measuring point number of inner server storage, and max_sig_id is the inner server
The maximum measuring point of storage is numbered, and serve_id is inner server number, and server_address is inner server access
Location；
Several server_element form inner server measuring point allocation table sig_server_list；
26) inner server measuring point allocation table sig_server_list is saved in memory server node information table.
3. the industrial signal data access method according to claim 1 based on fixed length block, which is characterized in that described
Measuring point is encoded on inner server the following steps are included:
31) configuration indexes in each inner server, and it includes single measuring point datas to index mono_ind_array, data
Block indexes block_ind_array；
Single measuring point data indexes the one-dimension array that mono_ind_array is address, and array index id is since 0, wherein subscript
The corresponding relationship of id and sig_id is
Id=sig_id-min_sig_id；
Data block index block_ind_array be two-dimensional array, wherein each element structure be pos, length,
address},
Pos is used to indicate current newest measuring point data storage position, and length is used to indicate entire data block length,
Address is used to indicate data block memory first address；
Queue is configured in each inner server, it includes the queues of single measuring point data and measuring point data block queue, wherein
Single measuring point data queue is for storing the last received measuring point data, and measuring point data block queue according to client for wanting
The measuring point data stored in nearest a period of time is sought, the distribution of measuring point data block is used by the data block length fixed size of setting
Independent memory headroom storage；
32) all inner server data are initialized, corresponding memory address are filled to measuring point index, successively to
The first address for the memory address that mono_ind_array, block_ind_array filling distribute.
4. the industrial signal data access method according to claim 1 based on fixed length block, which is characterized in that described
Reception data high-speed bus transmitted by data the following steps are included:
41) data be written scheduler from high speed data bus receive data, data mode be sig_name, timestamp,
Value }, sig_name is the hashed value of signal name, and timestamp is time tag, and value is measuring point signal value measuring point letter
Number numerical value got；
42) data write-in scheduler uses binary chop, and sig_name is searched from sig_id_array, obtains sig_name
Array index be sig_id；Sig_server_list, which is retrieved, according to sig_id finds inner server address；
43) data write-in scheduler sends the data to the inner server found.
5. the industrial signal data access method according to claim 1 based on fixed length block, which is characterized in that described
Inner server data reception and processing the following steps are included:
51) inner server receives data, is saved in single measuring point data；
I_cursor=sig_id-min_sig_id, which is designated as, under calculating single measuring point data index obtains measuring point in current server
Single-point index position；The memory address that measuring point data is stored is obtained at index mono_ind_array [i_cursor],
Data are written in this address；
52) inner server receives data, saves data to measuring point data block, steps are as follows:
Calculate i_cursor=sig_id-min_sig_id；Measuring point is obtained in the measuring point data block index position of current server；
Measuring point data storage data block first address block_address is obtained from block_ind_array [i_cursor], and
Obtain the current measuring point data storage position pos of data block, data block length len；
Calculate pos=pos mod len；In block_address [pos] storage measuring point signal data record；In current location
Cover newest measuring point signal data；
Block_ind_array [i_cursor] current location pos is updated to pos+1mod len.
6. the industrial signal data access method according to claim 1 based on fixed length block, which is characterized in that described
Request server or client read data packet include following steps:
61) client transmit a request to data access scheduler, request comprising sig_name and request data be single still
Data block,
According to the signal name sig_name of request, measuring point mark sig_id is found from sig_id_array；Check sig_
Server_list searches corresponding inner server, transmits the request to corresponding inner server；
If 62), request is reads single measuring point data, and inner server is from mono_ind_array [sig_id-min_sig_id]
Place directly reads data record and places extra buffer；
If 63) request is signal real-time curve, inner server is from block_address=block_ind_array [sig_
Id-min_sig_id] at directly read data block full content place extra buffer；
64) data block contents are arranged in extra buffer, steps are as follows:
After the data of block_address [0..pos-1] are exchanged with block_address [pos..len-1] data content
It is put into extra buffer；
65) encoder according to request server or client requirements by extra buffer data encoding at XML format or
JSON format, places output queue, and output queue is output to request server or client using FIFO mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710285624.6A CN107065800B (en) | 2017-04-27 | 2017-04-27 | Industrial signal data access method based on fixed length block |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710285624.6A CN107065800B (en) | 2017-04-27 | 2017-04-27 | Industrial signal data access method based on fixed length block |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107065800A CN107065800A (en) | 2017-08-18 |
| CN107065800B true CN107065800B (en) | 2019-04-09 |
Family
ID=59604787
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710285624.6A Active CN107065800B (en) | 2017-04-27 | 2017-04-27 | Industrial signal data access method based on fixed length block |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107065800B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101631061A (en) * | 2008-07-15 | 2010-01-20 | 中国移动通信集团公司 | Method for informing resource node message and network entity device |
| KR20120110776A (en) * | 2011-03-30 | 2012-10-10 | 주식회사 넥스프링 | Industrial wireless access point with control command setup features |
| CN102855239A (en) * | 2011-06-28 | 2013-01-02 | 清华大学 | Distributed geographical file system |
| CN102904923A (en) * | 2012-06-21 | 2013-01-30 | 华数传媒网络有限公司 | Data reading method and data reading system capable of relieving disk reading bottleneck |
| CN102932331A (en) * | 2012-09-29 | 2013-02-13 | 南京云创存储科技有限公司 | Super-safe-storage coding/decoding method applicable to distributed storage system |
| CN105095421A (en) * | 2015-07-14 | 2015-11-25 | 南京国电南自美卓控制系统有限公司 | Distributed storage method for real-time database |
| CN105228140A (en) * | 2015-08-31 | 2016-01-06 | 华为技术有限公司 | A kind of data access method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9923969B2 (en) * | 2015-08-05 | 2018-03-20 | Futurewei Technologies, Inc. | Large-scale storage and retrieval of data with well-bounded life |
-
2017
- 2017-04-27 CN CN201710285624.6A patent/CN107065800B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101631061A (en) * | 2008-07-15 | 2010-01-20 | 中国移动通信集团公司 | Method for informing resource node message and network entity device |
| KR20120110776A (en) * | 2011-03-30 | 2012-10-10 | 주식회사 넥스프링 | Industrial wireless access point with control command setup features |
| CN102855239A (en) * | 2011-06-28 | 2013-01-02 | 清华大学 | Distributed geographical file system |
| CN102904923A (en) * | 2012-06-21 | 2013-01-30 | 华数传媒网络有限公司 | Data reading method and data reading system capable of relieving disk reading bottleneck |
| CN102932331A (en) * | 2012-09-29 | 2013-02-13 | 南京云创存储科技有限公司 | Super-safe-storage coding/decoding method applicable to distributed storage system |
| CN105095421A (en) * | 2015-07-14 | 2015-11-25 | 南京国电南自美卓控制系统有限公司 | Distributed storage method for real-time database |
| CN105228140A (en) * | 2015-08-31 | 2016-01-06 | 华为技术有限公司 | A kind of data access method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107065800A (en) | 2017-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20170177646A1 (en) | Processing time series data from multiple sensors | |
| CN109951463A (en) | A kind of Internet of Things big data analysis method stored based on stream calculation and novel column | |
| CN202058147U (en) | Distribution type real-time database management system | |
| CN104252536B (en) | A kind of internet log data query method and device based on hbase | |
| CN104516979B (en) | A kind of data query method and system based on quadratic search | |
| US12019911B2 (en) | Storage management apparatus, method and program | |
| CN105608188A (en) | Data processing method and data processing device | |
| CN104252528B (en) | Big data secondary index establishing method based on identifier space mapping | |
| KR20150112357A (en) | Sensor data processing system and method thereof | |
| CN107302569B (en) | Cloud platform-oriented security monitoring data acquisition and storage method | |
| CN112749153B (en) | Industrial network data management system | |
| CN103207920A (en) | Parallel metadata acquisition system | |
| CN110427446B (en) | Method and system for rapidly publishing and browsing mass image services | |
| CN103886508A (en) | Mass farmland data monitoring method and system | |
| CN112905600B (en) | Data query method and device, storage medium and electronic equipment | |
| CN104199978A (en) | System and method for realizing metadata cache and analysis based on NoSQL and method | |
| CN102946320A (en) | Distributed supervision method and system for user behavior log forecasting network | |
| CN108133043B (en) | Structured storage method for server running logs based on big data | |
| CN110334290A (en) | A kind of space-time data method for quickly retrieving based on MF-Octree | |
| CN103198126B (en) | Internet of Things Temporal-spatial data management method | |
| CN103218396B (en) | The management and running visual analysis method of static Web page is generated according to visitation frequency feature | |
| CN107065800B (en) | Industrial signal data access method based on fixed length block | |
| CN112905571B (en) | Train rail transit sensor data management method and device | |
| CN117527805A (en) | Multi-mode data access optimization method and system based on server cluster | |
| KR20220071548A (en) | Sensor data processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Industrial signal data access method based on fixed length data block Effective date of registration: 20220324 Granted publication date: 20190409 Pledgee: China Construction Bank Corporation Hefei Shushan sub branch Pledgor: HEFEI CITY CLOUD DATA CENTER Co.,Ltd. Registration number: Y2022980003149 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |