CN107065800A - Industrial signal data access method based on fixed length block - Google Patents
Industrial signal data access method based on fixed length block Download PDFInfo
- Publication number
- CN107065800A CN107065800A CN201710285624.6A CN201710285624A CN107065800A CN 107065800 A CN107065800 A CN 107065800A CN 201710285624 A CN201710285624 A CN 201710285624A CN 107065800 A CN107065800 A CN 107065800A
- Authority
- CN
- China
- Prior art keywords
- data
- measuring point
- server
- sig
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000012545 processing Methods 0.000 claims abstract description 10
- 230000015654 memory Effects 0.000 claims description 54
- 238000003860 storage Methods 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 12
- 238000013500 data storage Methods 0.000 claims description 7
- 238000000151 deposition Methods 0.000 claims description 6
- 238000004321 preservation Methods 0.000 claims description 5
- 238000009826 distribution Methods 0.000 claims description 3
- 230000004083 survival effect Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 11
- 230000007547 defect Effects 0.000 abstract description 2
- 238000012544 monitoring process Methods 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 4
- 241001269238 Data Species 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000001747 exhibiting effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
- G05B19/4185—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM] characterised by the network communication
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/33—Director till display
- G05B2219/33139—Design of industrial communication system with expert system
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Computer And Data Communications (AREA)
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Manufacturing & Machinery (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to the industrial signal data access method based on fixed length block, the low defect for causing data to extract the wasting of resources of data user rate that industrial signal source is extracted is solved compared with prior art.The present invention comprises the following steps：Build inner server cluster；Measuring point is encoded on inner server；Receive the data transmitted by data high-speed bus；The reception and processing of inner server data；Request server or client read data.Present invention reduces the calculated load that server retrieves data, the technology key data for meeting client show and industrial curve form shows requirement in real time in real time.
Description
Technical field
The present invention relates to industrial data access technique field, the specifically industrial signal data based on fixed length block
Access method.
Background technology
It is perfect with the business such as Internet of Things, cloud computing, big data and technology, and the national strategy of made in China 2025 pushes away
It is dynamic, industrial data center as industrial data collect and store, Distribution Center, irreplaceable effect will be played.In work
, it is necessary to which industrial data center can efficiently collect discrete industrial measuring point data signal in the everyday scenes of industry application, and
Data can be efficiently retrieved, to meet the requirement of industry spot Real-time process data exhibiting and real-time curve form.And be
The high speed access of industrial implementation signal is met, it is using efficient data buffer storage administrative skill essential.
In industrial circle, the storage of live signal and retrieval scheme are a lot, and wherein typical scenario is typically via reality
When database especially RTDB in Industry Control come realize measuring point data cache.There is real-time data base high healthy and strong, height to handle up, thing
The features such as being engaged in safe, but because its is expensive, be often used in MES, MRP system of large enterprise, for data center this
Its cost is unacceptable for the demand that sample is provided infrastructures for conglomerate and is not necessarily to.
Also there are many technical schemes to use to build distributed memory data queue to solve work based on Memchched, Redis etc.
Industry data buffer storage problem, it is inadequate to carry out data storage only with K-Value modes, it is still necessary to build distributed data cache shelf
Structure and being actually needed according to business builds corresponding data structure and defines access strategy.Need it is to be noted that
The UCBerkeley distributed variable-frequencypump framework Spark based on internal memory and the Storm based on MapReduce is provided which
Data processing mechanism based on stream process, solves the real-time computational problem of big data, but slow as simple data
Rush and improper.
Internal memory computing technique in research field, big data Technology Chain is gradually paid close attention to by people in recent years, country in 2015
" towards the internal memory key technology of big data " is classified as the important content of areas of information technology by 863 Program.But Domestic Scientific Research
The scientific achievement of mechanism, which is partial to, emphasizes learning value, and achievements conversion has certain difficulty.With the manufacture of computer hardware equipment
The lifting of technology, and super integration technology popularization, using super integration technology framework or inexpensive vast capacity memory service
Device can bring new solution for the industrial data caching at industrial data center.
For the angle at industrial data center, industrial data center generally requires to converge substantial amounts of signal data.Assuming that
One group of industrial website of server access more than 1000, each industry website is needed to monitor 1000 signal measuring points, and each measuring point is needed
2000 datas of retrieval are preserved in real time, then server needs to provide preservation and retrieval service for nearly 200,000,000 records.Even if these
Data are all preserved using internal memory, if not being considered as suitable Data Structure and Algorithm, then server retrieval calculating pressure is huge
Greatly, it is impossible to the request of data of summary responses and processing high frequency time.From construction cost angle, data center also should not be only
Meet data buffering needs and use the high very large memory server of price.
From the point of view of the practical application of industrial remote monitoring, remote monitoring generally requires consideration observation process node key index
Data, and observe the real-time curve of a certain key index time zone.And the instant computing of live signal and in real time letter
Number backtracking often industrial environment scene and local information center processing.Therefore it is analyzed as follows：Remote monitoring is to technique
, it is necessary to observe measuring point latest data with reference to process structure figure, this is to need only to server to send to client when node is monitored
The measuring point wall scroll latest data；And when measuring point real-time curve is observed, display device resolution sizes are limited to, take
Business device needs only to send the measuring point data of regular length to client.
Therefore, the mass data extracted in conventional art is actual to be used in the then considerably less of client, causes data
The wasting of resources of extraction process.This data extraction property of industrial remote monitoring in actual applications how is utilized, work is reduced
The extracted amount of industry source point data, designs a kind of industrial signal data access method and has become the technical problem for being badly in need of solving.
The content of the invention
The invention aims to solve, industrial signal source is extracted in the prior art data user rate is low to cause data
The defect for extracting the wasting of resources solves above-mentioned ask there is provided a kind of industrial signal data access method based on fixed length block
Topic.
To achieve these goals, technical scheme is as follows：
A kind of industrial signal data access method based on fixed length block, comprises the following steps：
Inner server cluster is built, is needed to set data block length according to the generation of measuring point real-time curve, internal memory is taken
Business device carries out memory configurations, and sets up inner server informational table of nodes；
Measuring point is encoded on inner server, the measuring point stored respectively to it on every inner server is carried out
Encode and set up the Memory Allocation structure of inner server；
The data transmitted by data high-speed bus are received, data write-in scheduler receives the number transmitted by data high-speed bus
According to, and it is stored in data to inner server；
The reception and processing of inner server data, inner server receive the data that data write scheduler dispatches, and
Handled, update corresponding data block；
Request server or client read data, and client reads corresponding data, with generate measuring point real-time curve or
Show Real-time process data.
Described structure inner server cluster comprises the following steps：
The demand of measuring point real-time curve is generated according to client, set data block length as 128,256,512,1024 or
2048；
Mark sig_id is set up to measuring point, is identified from 0 open numbering；
Calculate every inner server memory size, the survey according to corresponding to data block length distributes every inner server
Points, and corresponding measuring point is assigned to every inner server；
Inner server informational table of nodes is set up, array sig_id_ is set up in internal memory server node information table
Array, array index are sig_id, array sig_id_array storage signal names and measuring point identity map, and signal name is
The corresponding measuring point of sig_name, sig_name is designated sig_id；
Set up and preserve the corresponding inner server information server_element of measuring point data access, inner server information
Server_element structures are as follows：
{min_sig_id,max_sig_id,server_id,server_address},
Wherein：Min_sig_id is the minimum measuring point numbering that the inner server is deposited, and max_sig_id takes for the internal memory
The maximum measuring point numbering of business device storage, serve_id is that inner server is numbered, and server_address is that inner server is visited
Ask address；
Several server_element composition inner server measuring point allocation tables sig_server_list；
Inner server measuring point allocation table sig_server_list is preserved in internal memory server node information table.
It is described coding is carried out on inner server to measuring point to comprise the following steps：
The equal configuration index in each inner server, it includes wall scroll measuring point data index mono_ind_array, number
Block_ind_array is indexed according to block；
Wall scroll measuring point data index mono_ind_array be address one-dimension array, array index id since 0, wherein
Subscript id and sig_id corresponding relation is
Id=sig_id-min_sig_id；
Data block index block_ind_array be two-dimensional array, wherein each element structure for pos, length,
address},
Pos is used to indicate current newest measuring point data deposit position, and length is used to indicate whole data block length,
Address is used to indicate data block internal memory first address；
Queue is configured in each inner server, it includes the queue of wall scroll measuring point data and measuring point data block queue,
Wherein the queue of wall scroll measuring point data is used for the measuring point data for depositing the last reception, and the queue of measuring point data block is used for according to client
End requires the measuring point data in storage nearest a period of time, and the data block length fixed size of setting is pressed in the distribution of measuring point data block,
Deposited using independent memory headroom；
All inner server data are initialized, to the corresponding memory address of measuring point index filling, successively to
The first address for the memory address that mono_ind_array, block_ind_array filling are distributed.
Data transmitted by described reception data high-speed bus comprise the following steps：
Data write scheduler from high speed data bus receive data, its data mode for sig_name, timestamp,
Value }, sig_name is the hashed value of signal name, and timestamp is time tag, and value believes for measuring point signal value measuring point
Number numerical value got；
Data write-in scheduler uses binary chop, and sig_name is searched from sig_id_array, sig_ is obtained
Name array index is sig_id；Sig_server_list is retrieved according to sig_id and finds inner server address；
The data is sent to the inner server found by data write-in scheduler.
The reception of described inner server data comprises the following steps with processing：
Inner server receives data, is saved in wall scroll measuring point data；
Calculate and be designated as i_cursor=sig_id-min_sig_id under wall scroll measuring point data index and obtain measuring point taking currently
The single-point index position of business device；The internal memory of measuring point data storage is obtained from index mono_ind_array [i_cursor]
Location, data are write in this address；
Inner server receives data, and preservation data to measuring point data block, step is as follows：
Calculate i_cursor=sig_id-min_sig_id；The measuring point data block that measuring point is obtained in current server is indexed
Position；
Measuring point data storage data block first address block_ is obtained from block_ind_array [i_cursor]
Address, and obtain data block current measuring point data deposit position pos, data block length len；
Calculate pos=pos mod len；In block_address [pos] storage measuring point signal data records；Current
Position covers newest measuring point signal data；
Block_ind_array [i_cursor] current location pos is updated to pos+1mod len.
Described request server or client read data packet includes following steps：
Client transmit a request to data access scheduler, and request bag contains sig_name, and request data be wall scroll also
It is data block,
According to the signal name sig_name of request, measuring point mark sig_id is found from sig_id_array；Check
Sig_server_list searches correspondence inner server, transmits the request to corresponding inner server；
If request is reads wall scroll measuring point data, inner server is from mono_ind_array [sig_id-min_sig_
Id] place directly reads data record and places extra buffer；
If request is signal real-time curve, inner server is from block_address=
Block_ind_array [sig_id-min_sig_id] place directly reads data block full content and places interim slow
Rush area；
Data block contents are arranged in extra buffer, step is as follows：
Block_address [0..pos-1] data and block_address [pos..len-1] data content are handed over
Extra buffer is put into after changing；
Encoder according to request server or client requirements by extra buffer data encoding into XML format or
JSON forms, place output queue, and output queue is output to request server or client using FIFO modes.
Beneficial effect
The industrial signal data access method based on fixed length block of the present invention, compared with prior art using industrial remote
The data characteristic of range monitoring uses the queue of wall scroll measuring point and data block queue storing data, and improves internal memory using indexed mode
Retrieval rate of the server to industrial signal.The present invention is all taken out the measuring point data of certain time sequence using indexed mode
And send, to reduce service retrieval load, so as to reduce the calculated load that server retrieves data, meet the work of client
Skill critical data shows in real time and the real-time of industrial curve form shows requirement.
Brief description of the drawings
Fig. 1 is method precedence diagram of the invention；
Fig. 2 is the memory allocation map of inner server in the present invention；
Fig. 3 accesses connection block diagram for industrial signal data in the prior art.
Embodiment
To make to have a better understanding and awareness to architectural feature of the invention and the effect reached, to preferably
Embodiment and accompanying drawing coordinate detailed description, are described as follows：
As shown in figure 3, the industrial signal data of prior art is accessed in annexation, high speed data bus is used for internal memory
Server group writes real-time measuring point data, can use ZMQ, RabbitMQ, ZeroMQ, JMS, MSMQ, ActiveMQ, Jafka/
Any one in Kafka is realized.
Inner server informational table of nodes is used to preserve inner server measuring point allocation table sig_server_list and array
sig_id_array。
Data write scheduler and read measuring point signal data from high speed data bus, and selection inner server preserves data.
Inner server is used to depositing and retrieving data, and its Memory Allocation graph structure is as shown in Fig. 2 wherein wall scroll measuring point
Data directory is used to deposit every measuring point data address, and measuring point data block index preserves measuring point data block first address and data
Block length, when presubscript information.Wall scroll measuring point data queue preserves each measuring point data record, and measuring point data record format is
{ sig_name, time_stamp, value }.Measuring point data block preserves the measuring point data record in certain time sequence, measuring point number
It is regular length according to block.
Data access scheduler receives request server or client data request, the service that selection measuring point signal is present
Device.What it is such as request is single technique measuring point signal data, then takes out wall scroll measuring point data record from corresponding inner server.As asked
What is asked is one section of real-time curve data, then all data records of whole data block are taken out from corresponding inner server, collated
After provide.
The data that encoder takes out to data access scheduling device, are encoded by request server or client requirements.
The measuring point data record that output caching is used to deposit after coding, waits request server or client consumption number
According to.
As shown in figure 1, the industrial signal data access method of the present invention based on fixed length block, including following step
Suddenly：
The first step, builds inner server cluster.Needed to set data block length according to the generation of measuring point real-time curve, it is right
Inner server carries out memory configurations, and sets up inner server informational table of nodes.Here, to reduce the time-consuming of data query,
Corresponding inner server is distributed for each industrial data node, and corresponding index is set up to inner server.Carrying out
During industrial node data inquiry, without inquiring about, traditional database or the industrial node of access, only need to access corresponding internal storage data
Storehouse.It is comprised the following steps that：
(1) demand of measuring point real-time curve is generated according to client, data block length is set as 128,256,512,1024
Or 2048, data block length is in units of data bit.
(2) mark sig_id is set up to measuring point (industrial signal node), is identified from 0 open numbering.
(3) every inner server memory size is calculated, according to corresponding to data block length distributes every inner server
Measuring point number, and corresponding measuring point is assigned to every inner server.For example：Amount to and be provided with 4 inner servers, every
32G internal memories can be provided for depositing data block by depositing server, if measuring point data block length is 1K, then every server can divide
It it is 32M with measuring point number.
(4) inner server informational table of nodes is set up.Array sig_id_ is set up in internal memory server node information table
Array, array are designated as sig_id, array sig_id_array storage signal names and measuring point identity map, signal name under being
For sig_name, the corresponding measuring points of sig_name are designated sig_id.Hash-coding is used for ease of quick-searching sig_name.
(5) set up and preserve the corresponding inner server information server_element of measuring point data access, inner server
Information server_element structures are as follows：
{min_sig_id,max_sig_id,server_id,server_address},
Wherein：Min_sig_id is the minimum measuring point numbering that the inner server is deposited, and max_sig_id takes for the internal memory
The maximum measuring point numbering of business device storage, serve_id is that inner server is numbered, and server_address is that inner server is visited
Ask address.
Several server_element composition inner server measuring point allocation tables sig_server_list.
(6) inner server measuring point allocation table sig_server_list is preserved in internal memory server node information table.
Second step, is encoded to measuring point on inner server.It is stored respectively on every inner server
Measuring point is encoded and sets up the Memory Allocation structure of inner server.During for accessing inner server, also simultaneously
It is unconventional to be searched for inner server, but take and directly transferred from corresponding inner server (non-to search)
Method.For in actual use, it is necessary to reference to process structure figure observe measuring point latest data data transfer mode, only need read
Wall scroll measuring point data queue, and in wall scroll measuring point data queue it is the number for the single-point (a certain industrial signal source) constantly updated
According to, then be equipped with wall scroll measuring point data index positioned；And when need to generate measuring point real-time curve, then need to read measuring point data block team
Row, preserved in measuring point data block queue for the data in single-point (a certain industrial signal source) data block length, then be equipped with data
Block index is positioned, so as to reduce calculating and the response time of system, improves operational efficiency.It is comprised the following steps that：
(1) the equal configuration index in each inner server, its comprising wall scroll measuring point data index mono_ind_array,
Data block indexes block_ind_array, and wall scroll measuring point data index mono_ind_array is the one-dimension array of an address.
Array index id is since 0.Wherein subscript id and sig_id corresponding relation is:Id=sig_id-min_sig_id；Data block
It is a two-dimensional array to index block_ind_array, wherein each element structure is { pos, length, address }, pos
For indicating current newest measuring point data deposit position, length is used to indicate whole data block length, and address is used to refer to
Show data block internal memory first address.
Queue is configured in each inner server, it includes the queue of wall scroll measuring point data and measuring point data block queue,
Wherein the queue of wall scroll measuring point data is used for the measuring point data for depositing the last reception, and the queue of measuring point data block is used for according to client
End requires the measuring point data in storage nearest a period of time, and the data block length fixed size of setting is pressed in the distribution of measuring point data block,
Deposited using independent memory headroom.
(2) all inner server data are initialized, memory address corresponding to measuring point index filling.Specifically
Successively to mono_ind_array, the first address for the memory address that block_ind_array fillings are distributed.
3rd step, receives the data transmitted by data high-speed bus.Data write-in scheduler receives data high-speed bus institute
The data of transmission, and it is stored in data to inner server.It is comprised the following steps that：
(1) data write scheduler from high speed data bus receive data, its data mode for sig_name,
Timestamp, value }, wherein：Sig_name is the hashed value of signal name, and timestamp is time tag, and value is
The numerical value that measuring point signal value measuring point signal is got.
(2) data write-in scheduler uses binary chop, sig_name is searched from sig_id_array, by the first step
(4) understand, correspondence sig_name array index is sig_id；Retrieved according to sig_id in sig_server_list finds
Deposit server address.
(3) data is sent to the inner server found by data write-in scheduler.
It can thus be seen that the present invention simply writes the lookup of progress inner server in scheduler in data, i.e., it is traditional
Industrial signal big data source lookup, be changed into carrying out the lookup of inner server herein, find out required memory service
Device address, greatly reduces data search amount.
4th step, the reception and processing of inner server data.Inner server receives data write-in scheduler dispatches
Data, and handled, update corresponding data block.In view of each measuring point data may be used as technology key index or
It is as a process curve part, it is therefore desirable to preserve, comprise the following steps that simultaneously：
(1) inner server receives data, is saved in wall scroll measuring point data；
Calculate and be designated as i_cursor=sig_id-min_sig_id under wall scroll measuring point data index and obtain measuring point taking currently
The single-point index position of business device；The internal memory of measuring point data storage is obtained from index mono_ind_array [i_cursor]
Location, data are write in this address；
(2) inner server receives data, and preservation data to measuring point data block, step is as follows：
Calculate i_cursor=sig_id-min_sig_id；The measuring point data block that measuring point is obtained in current server is indexed
Position；
Measuring point data storage data block first address block_ is obtained from block_ind_array [i_cursor]
Address, and obtain data block current measuring point data deposit position pos, data block length len；
Calculate pos=pos mod len；In block_address [pos] storage measuring point signal data records；Current
Position covers newest measuring point signal data.
Block_ind_array [i_cursor] current location pos is updated to pos+1mod len.It is next for indicating
Bar measuring point data deposit position.
Here, inner server constantly carries out the renewal of data, to wait request server or client to carry out data reading
Take.
Thus mode, can constantly refresh in the data block internal memory of fixed size, preserves the survey in nearest a period of time
Point signal data.
5th step, request server or client read data.Client reads corresponding data, bent in real time to generate measuring point
Line or displaying Real-time process data.It is comprised the following steps that：
(1) client transmit a request to data access scheduler, and request bag contains sig_name, and request data is wall scroll
Or data block.
According to the signal name sig_name of request, measuring point mark sig_id is found from sig_id_array；Check
Sig_server_list searches correspondence inner server, transmits the request to corresponding inner server.
(2) if request is reads wall scroll measuring point data, inner server is from mono_ind_array [sig_id-min_
Sig_id] place directly reads data record and places extra buffer.
(3) if request is signal real-time curve, inner server is from block_address=block_ind_array
[sig_id-min_sig_id] place directly reads data block full content and places extra buffer；
(4) data block contents are arranged in extra buffer,
Step is as follows：
Block_address [0..pos-1] data and block_address [pos..len-1] data content are handed over
Extra buffer is put into after changing, it is ensured that data block contents are to be sequentially arranged；
(5) encoder according to request server or client requirements by extra buffer data encoding into XML format or
Person's JSON forms, place output queue, and output queue is output to request server or client using FIFO modes.
In industry spot monitoring system, it is desirable to be able to show in real time live measuring point data critical process index or
Show the real-time curve in nearest a period of time.And the preservation that the industrial data center serviced for it is required to high speed is substantial amounts of
Real-time live signal data quickly interviewee can simultaneously be retrieved.The present invention uses inner server measuring point allocation table and measuring point
Index the different inner servers and memory address that two-layer configuration indicates unlike signal data access.The present invention uses wall scroll
Measuring point queue and data block queue preserve signal data, and the queue of wall scroll measuring point is used to store newest one-time process node key index
Data, data block queue preserves the process node measuring point signal data in nearest a period of time, as a result of directly covering side
The memory mechanism of formula, therefore do not save historical data.Using the access mode of as above mechanism, visitor is set directly to be taken using internal memory
It is engaged in device measuring point allocation table and measuring point index obtains data, it is to avoid the retrievals of convential memory data, greatly reduces memory service
The retrieval pressure of device, is adapted to the field that industrial data center handles a large amount of industrial signal datas using distributed memory server centered
Scape.
General principle, principal character and the advantages of the present invention of the present invention has been shown and described above.The technology of the industry
Personnel are it should be appreciated that the present invention is not limited to the above embodiments, and that described in above-described embodiment and specification is the present invention
Principle, various changes and modifications of the present invention are possible without departing from the spirit and scope of the present invention, these change and
Improvement is both fallen within the range of claimed invention.The protection domain of application claims by appended claims and its
Equivalent is defined.
Claims (6)
1. a kind of industrial signal data access method based on fixed length block, it is characterised in that comprise the following steps：
11) inner server cluster is built, is needed to set data block length according to the generation of measuring point real-time curve, to memory service
Device carries out memory configurations, and sets up inner server informational table of nodes；
12) measuring point is encoded on inner server, the measuring point stored respectively to it on every inner server is carried out
Encode and set up the Memory Allocation structure of inner server；
13) data transmitted by data high-speed bus are received, data write-in scheduler receives the number transmitted by data high-speed bus
According to, and it is stored in data to inner server；
14) reception and processing of inner server data, inner server receives the data that data write scheduler dispatches, goes forward side by side
Row processing, updates corresponding data block；
15) request server or client read data, and client reads corresponding data, to generate measuring point real-time curve or exhibition
Show Real-time process data.
2. the industrial signal data access method according to claim 1 based on fixed length block, it is characterised in that described
Structure inner server cluster comprise the following steps：
21) demand of measuring point real-time curve is generated according to client, set data block length as 128,256,512,1024 or
2048；
22) mark sig_id is set up to measuring point, is identified from 0 open numbering；
23) every inner server memory size, the survey according to corresponding to data block length distributes every inner server are calculated
Points, and corresponding measuring point is assigned to every inner server；
24) set up inner server informational table of nodes, set up in internal memory server node information table array sig_id_array,
Array index is sig_id, array sig_id_array storage signal names and measuring point identity map, and signal name is sig_
The corresponding measuring point of name, sig_name is designated sig_id；
25) set up and preserve the corresponding inner server information server_element of measuring point data access, inner server information
Server_element structures are as follows：
{min_sig_id,max_sig_id,server_id,server_address},
Wherein：Min_sig_id is the minimum measuring point numbering that the inner server is deposited, and max_sig_id is the inner server
The maximum measuring point numbering of storage, serve_id numbers for inner server, and server_address is inner server access
Location；
Several server_element composition inner server measuring point allocation tables sig_server_list；
26) inner server measuring point allocation table sig_server_list is preserved in internal memory server node information table.
3. the industrial signal data access method according to claim 1 based on fixed length block, it is characterised in that described
Coding carried out on inner server to measuring point comprised the following steps：
31) the equal configuration index in each inner server, it includes wall scroll measuring point data index mono_ind_array, data
Block indexes block_ind_array；
Wall scroll measuring point data index mono_ind_array is the one-dimension array of address, and array index id is since 0, wherein subscript
Id and sig_id corresponding relation is
Id=sig_id-min_sig_id；
Data block index block_ind_array be two-dimensional array, wherein each element structure for pos, length,
address},
Pos is used to indicate current newest measuring point data deposit position, and length is used to indicate whole data block length,
Address is used to indicate data block internal memory first address；
Queue is configured in each inner server, it includes the queue of wall scroll measuring point data and measuring point data block queue, wherein
The queue of wall scroll measuring point data is used for the measuring point data for depositing the last reception, and the queue of measuring point data block is used for will according to client
Seek survival the measuring point data put in nearest a period of time, the distribution of measuring point data block is used by the data block length fixed size of setting
Independent memory headroom storage；
32) all inner server data are initialized, to the corresponding memory address of measuring point index filling, successively to
The first address for the memory address that mono_ind_array, block_ind_array filling are distributed.
4. the industrial signal data access method according to claim 1 based on fixed length block, it is characterised in that described
Reception data high-speed bus transmitted by data comprise the following steps：
41) data write scheduler from high speed data bus receive data, its data mode for sig_name, timestamp,
Value }, sig_name is the hashed value of signal name, and timestamp is time tag, and value believes for measuring point signal value measuring point
Number numerical value got；
42) data write-in scheduler uses binary chop, and sig_name is searched from sig_id_array, sig_name is obtained
Array index be sig_id；Sig_server_list is retrieved according to sig_id and finds inner server address；
43) data is sent to the inner server found by data write-in scheduler.
5. the industrial signal data access method according to claim 1 based on fixed length block, it is characterised in that described
The receptions of inner server data comprise the following steps with processing：
51) inner server receives data, is saved in wall scroll measuring point data；
I_cursor=sig_id-min_sig_id, which are designated as, under calculating wall scroll measuring point data index obtains measuring point in current server
Single-point index position；The memory address of measuring point data storage is obtained from index mono_ind_array [i_cursor],
This address writes data；
52) inner server receives data, and preservation data to measuring point data block, step is as follows：
Calculate i_cursor=sig_id-min_sig_id；Obtain measuring point data block index position of the measuring point in current server；
Measuring point data storage data block first address block_address is obtained from block_ind_array [i_cursor], and
Obtain data block current measuring point data deposit position pos, data block length len；
Calculate pos=pos mod len；In block_address [pos] storage measuring point signal data records；In current location
The newest measuring point signal data of covering；
Block_ind_array [i_cursor] current location pos is updated to pos+1mod len.
6. the industrial signal data access method according to claim 1 based on fixed length block, it is characterised in that described
Request server or client read data packet include following steps：
61) client transmit a request to data access scheduler, and request bag contains sig_name, and request data be wall scroll still
Data block,
According to the signal name sig_name of request, measuring point mark sig_id is found from sig_id_array；Check sig_
Server_list searches correspondence inner server, transmits the request to corresponding inner server；
If 62), request is reads wall scroll measuring point data, and inner server is from mono_ind_array [sig_id-min_sig_id]
Place directly reads data record and places extra buffer；
If 63) request is signal real-time curve, inner server is from block_address=block_ind_array [sig_
Id-min_sig_id] place directly read data block full content place extra buffer；
64) data block contents are arranged in extra buffer, step is as follows：
After block_address [0..pos-1] data are exchanged with block_address [pos..len-1] data content
It is put into extra buffer；
65) encoder according to request server or client requirements by extra buffer data encoding into XML format or
JSON forms, place output queue, and output queue is output to request server or client using FIFO modes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710285624.6A CN107065800B (en) | 2017-04-27 | 2017-04-27 | Industrial signal data access method based on fixed length block |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710285624.6A CN107065800B (en) | 2017-04-27 | 2017-04-27 | Industrial signal data access method based on fixed length block |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107065800A true CN107065800A (en) | 2017-08-18 |
| CN107065800B CN107065800B (en) | 2019-04-09 |
Family
ID=59604787
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710285624.6A Active CN107065800B (en) | 2017-04-27 | 2017-04-27 | Industrial signal data access method based on fixed length block |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107065800B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101631061A (en) * | 2008-07-15 | 2010-01-20 | 中国移动通信集团公司 | Method for informing resource node message and network entity device |
| KR20120110776A (en) * | 2011-03-30 | 2012-10-10 | 주식회사 넥스프링 | Industrial wireless access point with control command setup features |
| CN102855239A (en) * | 2011-06-28 | 2013-01-02 | 清华大学 | Distributed geographical file system |
| CN102904923A (en) * | 2012-06-21 | 2013-01-30 | 华数传媒网络有限公司 | Data reading method and data reading system capable of relieving disk reading bottleneck |
| CN102932331A (en) * | 2012-09-29 | 2013-02-13 | 南京云创存储科技有限公司 | Super-safe-storage coding/decoding method applicable to distributed storage system |
| CN105095421A (en) * | 2015-07-14 | 2015-11-25 | 南京国电南自美卓控制系统有限公司 | Distributed storage method for real-time database |
| CN105228140A (en) * | 2015-08-31 | 2016-01-06 | 华为技术有限公司 | A kind of data access method and device |
| US20170041394A1 (en) * | 2015-08-05 | 2017-02-09 | Futurewei Technologies, Inc. | Large-Scale Storage and Retrieval of Data with Well-Bounded Life |
-
2017
- 2017-04-27 CN CN201710285624.6A patent/CN107065800B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101631061A (en) * | 2008-07-15 | 2010-01-20 | 中国移动通信集团公司 | Method for informing resource node message and network entity device |
| KR20120110776A (en) * | 2011-03-30 | 2012-10-10 | 주식회사 넥스프링 | Industrial wireless access point with control command setup features |
| CN102855239A (en) * | 2011-06-28 | 2013-01-02 | 清华大学 | Distributed geographical file system |
| CN102904923A (en) * | 2012-06-21 | 2013-01-30 | 华数传媒网络有限公司 | Data reading method and data reading system capable of relieving disk reading bottleneck |
| CN102932331A (en) * | 2012-09-29 | 2013-02-13 | 南京云创存储科技有限公司 | Super-safe-storage coding/decoding method applicable to distributed storage system |
| CN105095421A (en) * | 2015-07-14 | 2015-11-25 | 南京国电南自美卓控制系统有限公司 | Distributed storage method for real-time database |
| US20170041394A1 (en) * | 2015-08-05 | 2017-02-09 | Futurewei Technologies, Inc. | Large-Scale Storage and Retrieval of Data with Well-Bounded Life |
| CN105228140A (en) * | 2015-08-31 | 2016-01-06 | 华为技术有限公司 | A kind of data access method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107065800B (en) | 2019-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103488704B (en) | A kind of date storage method and device | |
| CN104252536B (en) | A kind of internet log data query method and device based on hbase | |
| US20170177646A1 (en) | Processing time series data from multiple sensors | |
| CN103714134B (en) | Network flow data index method and system | |
| CN110825733B (en) | Multi-sampling-stream-oriented time series data management method and system | |
| CN109951463A (en) | A kind of Internet of Things big data analysis method stored based on stream calculation and novel column | |
| CN105608188A (en) | Data processing method and data processing device | |
| CN103020078B (en) | Distributing real-time data bank data hierarchy indexing means | |
| CN105868411A (en) | Non-relation type database and relation type database integrated data query method and system | |
| CN106980669A (en) | Data storage and acquisition method and device | |
| CN103646051B (en) | Big-data parallel processing system and method based on column storage | |
| CN104516979A (en) | Data query method and data query system based on quadratic search | |
| CN102402586A (en) | Distributed data storage method | |
| CN100458784C (en) | Researching system and method used in digital labrary | |
| CN101841438B (en) | Method or system for accessing and storing stream records of massive concurrent TCP streams | |
| CN104252457B (en) | A kind of method and apparatus for being managed to data acquisition system | |
| CN105930345A (en) | Hierarchical indexing method based on distributed real-time database system (DRTDBS) | |
| CN104199978A (en) | System and method for realizing metadata cache and analysis based on NoSQL and method | |
| CN107908357A (en) | Name data network Forwarding plane PIT storage organizations and its data retrieval method | |
| CN107193494A (en) | RDD (remote data description) persistence method based on SSD (solid State disk) and HDD (hard disk drive) hybrid storage system | |
| CN110334290A (en) | A kind of space-time data method for quickly retrieving based on MF-Octree | |
| CN103198126B (en) | Internet of Things Temporal-spatial data management method | |
| CN109165096A (en) | The caching of web cluster utilizes system and method | |
| US10204110B2 (en) | Method and system for deleting obsolete files from a file system | |
| CN103220379A (en) | Domain name reverse-resolution method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Industrial signal data access method based on fixed length data block Effective date of registration: 20220324 Granted publication date: 20190409 Pledgee: China Construction Bank Corporation Hefei Shushan sub branch Pledgor: HEFEI CITY CLOUD DATA CENTER Co.,Ltd. Registration number: Y2022980003149 |