CN110535965A - A kind of data processing method and device, storage medium - Google Patents
A kind of data processing method and device, storage medium Download PDFInfo
- Publication number
- CN110535965A CN110535965A CN201910831499.3A CN201910831499A CN110535965A CN 110535965 A CN110535965 A CN 110535965A CN 201910831499 A CN201910831499 A CN 201910831499A CN 110535965 A CN110535965 A CN 110535965A
- Authority
- CN
- China
- Prior art keywords
- data
- cloud
- cache
- acquisition equipment
- sent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application provides a kind of data processing method and device, storage medium.The data processing method is applied to Border Gateway, comprising: receives the data that acquisition equipment is sent, the data are to be sent to the data in cloud；Determine the currently network state with the cloud；When determining the network state is abnormality, the current cache information of the corresponding spatial cache of the acquisition equipment is obtained；According to the current cache information by the data buffer storage into the spatial cache.The data processing method can guarantee data integrity.
Description
Technical field
This application involves internet of things field, in particular to a kind of data processing method and device, storage medium.
Background technique
With the development of Internet of Things industry, the type of terminal device it is more and more, equipment supplier is more and more various, is
It is full that the real-time traffic demands of internet of things equipment are provided, accelerate the response and processing speed of the data of equipment, edge calculations scheme
It comes into being, computing capability has been distributed to the closer edge side of the terminals such as range sensor, pole from cloud by the appearance of edge calculations
The big computation burden for reducing network delay and cloud, edge calculations can be run in unclouded situation, but cloud
Platform is still essential part, and the data of edge calculations equipment long-term acquisition are still needed and are at a time sent
To cloud, to keep the integrality and continuity of the data of equipment, in order to which the behavior to equipment is further analyzed.
The core competence of edge calculations is the parsing to data, polymerize, and calculates, storage.In Internet of Things industry, node is set
The data of standby acquisition are the cores of business.After existing edge calculations scheme interior joint data reach Border Gateway, Border Gateway exists
After pre-processing to data, it is sent to cloud, but when Network status is poor or the temporary delay machine in cloud the case where, data exist
When being sent to cloud after the pretreatment of edge calculations gateway because it is possible that the case where loss of data the reason of network, thus
It cannot be guaranteed that the integrality of the data finally uploaded.
Summary of the invention
The embodiment of the present application is designed to provide a kind of data processing method and device, storage medium, to guarantee number
According to final integrality.
In a first aspect, the embodiment of the present application provides a kind of data processing method, it is applied to Border Gateway, comprising: reception is adopted
Collect the data that equipment is sent, the data are to be sent to the data in cloud；Determine the currently network state with the cloud；In
When determining that the network state is abnormality, the current cache information of the corresponding spatial cache of the acquisition equipment is obtained；Root
According to the current cache information by the data buffer storage into the spatial cache.
In the embodiment of the present application, before the data received are sent to cloud, can first determine currently with cloud
Data are not transferred directly to cloud when it is abnormal for determining network state by network state, but according to current cache information
Data are first cached in spatial cache.Compared with prior art, the network state in gateway and cloud occurs abnormal, and data can flow
To processing locality (in the spatial cache for being cached to gateway), be avoided that and upload data in Network Abnormal, so avoid due to
Loss of data caused by network cause, guarantees data integrity.
As a kind of possible implementation, the cache information includes buffer data size and cache threshold, according to described
Current cache information is by the data buffer storage into the spatial cache, comprising:
Whether the buffer data size for judging presently described acquisition equipment is more than the cache threshold, in the buffer data size
When more than the cache threshold, the data cached of the acquisition equipment for meeting default deletion condition is deleted；The data are delayed
It is stored in the corresponding spatial cache of the acquisition equipment.
In the embodiment of the present application, before data cached, it is contemplated that, cannot be immoderate the problem of gateway memory space
Caching is carried out to disk, acquires whether the corresponding buffer memory of equipment is more than cache threshold by first judging, if if being more than,
With regard to deleting a part, formerly data cache current data again, gateway storage pressure can be prevented excessive and the feelings such as failure
Condition, and then guarantee the stability of gateway caches data.
As a kind of possible implementation, in the current cache information for obtaining the corresponding spatial cache of the acquisition equipment
Before, the method also includes:
Determine itself corresponding multiple acquisition equipment；The time of data is respectively uploaded according to multiple acquisition equipment
It is spaced the cache threshold that the corresponding spatial cache of each acquisition equipment is determined with the data volume for uploading data.
In the embodiment of the present application, the time interval of data is uploaded according to acquisition equipment and uploads the data volume energy ratio of data
Cache threshold is relatively reasonably set, the caching pressure of gateway is reduced.
As a kind of possible implementation, the data cached of the acquisition equipment for meeting default deletion condition is deleted,
Include:
Determine that the time cached in the spatial cache of the acquisition equipment is greater than the first data cached of preset value；It deletes
It is described formerly data cached.
In the embodiment of the present application, when deleting data cached, the time for deleting caching is greater than the inline cache of preset value
Data, be equivalent to the very long data of cache-time to delete, this partial data since the time of caching is longer, data itself
Use value is little, can avoid deleting important data to a certain extent.
As a kind of possible implementation, in the current cache information for obtaining the corresponding spatial cache of the acquisition equipment
Before, the method also includes:
Receive the caching rule for multiple acquisition equipment corresponding with itself that the cloud is sent；The caching rule
It is used to indicate the spatial cache method of salary distribution of each acquisition equipment；Each acquisition equipment point is distributed according to the caching rule
Not corresponding spatial cache.
In the embodiment of the present application, caching rule is sent to Border Gateway by cloud, gateway is further according to caching rule distribution
Spatial cache can quickly and effectively complete the distribution of spatial cache.In addition to this, each acquisition equipment has corresponding caching
Space avoids influencing each other for the data of different acquisition equipment when data cached.
As a kind of possible implementation, the currently network state with the cloud is determined, comprising:
Whether detection currently can normally access network；When being currently able to normally access network, sent out according to predetermined period
Send heartbeat message to the cloud；In the heartbeat response message corresponding with the heartbeat message for receiving the cloud transmission
When, determine that the network state is normal condition；When not receiving the heartbeat response message that the cloud is sent, determine
The network state is the abnormality.
In the embodiment of the present application, it by the connection status between heartbeat message detection and cloud, can detect in real time
It is whether normal with the network state in cloud.
As a kind of possible implementation, according to the current cache information by the data buffer storage to the caching
After in space, the method also includes:
When determining that the network state reverts to normal condition, the data are sent to the cloud according to preset rules
End.
In the embodiment of the present application, after temporarily being cached data, however, it is determined that restore just with the network state in cloud
Often, data are uploaded to cloud again, guarantee data can also be timely transmitted to cloud while complete.
As a kind of possible implementation, the data are sent to the cloud according to preset rules, comprising:
According to the data volume size of the data by the data in batches be sent to the cloud.
In the embodiment of the present application, by by the data of caching in batches be sent to cloud, reduce and service bandwidth produced
Raw pressure can be avoided the data disposably uploaded and fail too much.
Second aspect, the embodiment of the present application provide a kind of data processing equipment, are applied to Border Gateway, data processing dress
Set each function including method described in any possible implementation for realizing first aspect and first aspect
Module.
The third aspect, the embodiment of the present application provide a kind of readable storage medium storing program for executing, meter are stored on the readable storage medium storing program for executing
Calculation machine program is executed when the computer program is run by computer as first aspect and first aspect any possible
Step in method described in implementation.
Detailed description of the invention
Technical solution in ord to more clearly illustrate embodiments of the present application will make below to required in the embodiment of the present application
Attached drawing is briefly described, it should be understood that the following drawings illustrates only some embodiments of the application, therefore should not be seen
Work is the restriction to range, for those of ordinary skill in the art, without creative efforts, can be with
Other relevant attached drawings are obtained according to these attached drawings.
Fig. 1 is limbic system structural schematic diagram provided by the embodiments of the present application.
Fig. 2 is data processing method flow chart provided by the embodiments of the present application.
Fig. 3 is the implementation flow chart that opposite side hoddy provided by the embodiments of the present application puts row configuration into.
Fig. 4 is the whole implementation flow chart of data processing method provided by the embodiments of the present application.
Fig. 5 is the functional block diagram of data processing equipment provided by the embodiments of the present application.
Icon: 100- limbic system；The cloud 101-；102- Border Gateway；103- acquires equipment；300- data processing dress
It sets；301- receiving module；302- processing module；303- obtains module.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application is described.
Fig. 1 is please referred to, Fig. 1 is the structural schematic diagram of limbic system 100 provided by the embodiments of the present application, limbic system 100
Applied to Internet of Things Network Communication, as shown in Figure 1, limbic system 100 includes cloud 101, multiple Border Gateway 102 and multiple acquisitions
Equipment 103.Wherein, cloud 101 communicates to connect respectively with multiple Border Gateway 102, and each Border Gateway 102 is corresponding with itself
One or more acquisition equipment 103 communicates to connect.
It is appreciated that cloud 101 and Border Gateway 102 are one-to-many relationship, Border Gateway 102 and acquisition equipment 103
Or one-to-many relationship can be one or more acquisition equipment 103 corresponding to Border Gateway 102
It is just configured when establishing the limbic system 100, such as some acquisition equipment 103 only connects with a specific Border Gateway 102
It connects, cannot be connect with other Border Gateway 102, it is chaotic to avoid the transmission between data.In addition to this kind of embodiment, may be used also
To be the identification information for being stored with acquisition equipment 103 corresponding with itself on Border Gateway 102, in multiple acquisition equipment 103
When sending data, the data of the transmission of acquisition equipment 103 of the corresponding identification information of Border Gateway 102 receptions.Certainly, in order to whole
The transmission confusion reigned of data in a limbic system 100 can also be that any other Border Gateway 102 that can be realized is adopted with multiple
Collect the embodiment of the corresponding relationship of equipment 103.
For cloud 101, it can be understood as the Cloud Server in limbic system 100, the data in limbic system 100 are final
Storage is beyond the clouds on 101, and certainly, cloud 101 is also provided with the platform for interacting with user, such as with cloud 101
The client of connection, user, can also be by clients to cloud by the data that store on the available cloud 101 of client
The relevant instruction of 101 transmission or demand.
For Border Gateway 102, it can be understood as platform or equipment with edge calculations ability.Edge calculations refer to
In the side close to object or data source header, the open platform being integrated using network, calculating, storage, application core ability, nearby
Most proximal end service is provided.The application program of edge calculations is initiated in edge side, is generated faster web services response, is met industry
Real time business, using intelligence, security and privacy in terms of primary demand.Edge calculations are in physical entity and industry
Between connection, or the top in physical entity.And cloud calculates, the historical data of still accessible edge calculations.At this
Apply in embodiment, Border Gateway 102 is the equal of the top of each acquisition equipment 103, for beyond the clouds 101 and acquisition equipment
Data forwarding is carried out between 103, is calculated with reducing cloud 101, the pressure of processing data, the stability of improve data transfer.
For acquiring equipment 103, it can be understood as all kinds of acquisition equipment, acquisition equipment are used to acquire data, such as respectively
Class sensor, water level sensor, temperature sensor etc., the acquisition layout of equipment 103 is in the point for needing to carry out data acquisition.Citing comes
It says, it is assumed that acquisition equipment 103 is water level sensor, is acquired with the real time water level of Mr. Yu's reservoir, then each acquisition equipment 103 can
The each position for needing to carry out real time water level acquisition in the reservoir is arranged in.The data that acquisition equipment 103 acquires are sent in real time
To Border Gateway 102, cloud is sent the data to by Border Gateway 102.
Data processing method provided by the embodiments of the present application is applied to limbic system 100, following jointing edge system 100
The data processing method of the embodiment of the present application is introduced.
It referring to figure 2., is the data processing method flow chart provided by the embodiments of the present application applied to Border Gateway 102,
As shown in Fig. 2, the data processing method includes:
Step 201: Border Gateway 102 receives the data that acquisition equipment 103 is sent.Wherein, the number that acquisition equipment 103 is sent
According to the data to be sent to cloud 101.
Step 202: Border Gateway 102 determines the currently network state with cloud 101.
Step 203: it is corresponding slow to obtain acquisition equipment 103 when determining network state is abnormality for Border Gateway 102
Deposit the current cache information in space.
Step 204: Border Gateway 102 according to current cache information by data buffer storage into spatial cache.
In the embodiment of the present application, Border Gateway 102 is after receiving the data that acquisition equipment 103 is sent, and first determination is worked as
Data are not transferred directly to cloud 101 when determining network state is abnormality by the preceding network state with cloud 101, and
It is first to be cached to data in spatial cache according to current cache information.Compared with prior art, Border Gateway 102 and cloud
There is exception in 101 network state, and data can flow to processing locality (being cached in the spatial cache of Border Gateway 102), energy
It avoids uploading data in Network Abnormal, and then avoids the loss of data as caused by network cause, guarantee data integrity.
It should be noted that not limiting the quantity of acquisition equipment 103, in step 201- step 204 in this Shen
Please embodiment data processing method implementation process in, in step 201, the data received in synchronization may be more than
The data that equipment 103 is sent are acquired for one, but for the data that each acquisition equipment 103 is sent, Border Gateway 102
It is all handled by the way of step 201- step 204, therefore, step 201- step 204 is suitable for Border Gateway 102 to every
The data that a acquisition equipment 103 is sent are handled, and the data handling procedure between each acquisition equipment 103 will not mutual shadow
It rings.Next the embodiment of each step in step 201- step 204 is described.
In step 201, the data that acquisition equipment 103 is sent can be real-time data, be also possible to non real-time number
According to.If real-time data, then acquisition equipment 103 is transmitted to Border Gateway 102 after often collecting a data, such as certain
The waterlevel data at one moment.If non real-time data, then acquisition equipment 103 can periodically send collected number
According to such as at interval of the data of transmission in 5 minutes, the data sent every time are the data acquired in this gap periods.If
The mode of periodicity sending data, the period for sending data can be according to the change frequency of the acquisition data collected of equipment 103
It is configured.For example, can be set at least one hour as the frequency that waterlevel data changes is relatively low, as temperature data becomes
The frequency of change is relatively high, can be set at least 10 minutes etc..In addition, in this embodiment, multiple data are had sent,
In each data corresponding acquisition moment be different, can be by data and its when acquiring equipment 103 and sending the data of batch
The corresponding acquisition moment sends together.
After executing the step 201, Border Gateway 102 executes step 202, and for step 202, the embodiment of the present application is provided
A kind of optional embodiment, step 202 include: that Border Gateway 102 detects currently whether can normally access network；Current
When can normally access network, Border Gateway 102 sends heartbeat message to cloud 101 according to predetermined period；Receiving cloud
When the heartbeat response message corresponding with heartbeat message of 101 transmissions, determine that network state is normal condition；It is described not receiving
When the heartbeat response message that cloud is sent, determine that network state is abnormality.
In this embodiment, the purpose that whether detection currently can normally access network is determining current edge gateway
102 itself can connect network, if itself cannot connect network, can not realize detection and gateway network state whether
Normally, while also the misjudgment for leading to network state due to cannot connecting network at itself is avoided.Border Gateway 102 can
To pass through the WIFI for running relevant Programmable detection Border Gateway 102, the nets such as local area network (LAN, Local Area Network)
The state of card, to determine whether normally to access network.When confirmation normally can access network, Border Gateway 102 and cloud
User Datagram Protocol (UDP, User Datagram Protocol) or Websocket connection are established in end 101, can pass through
The link that the mode in the heartbeat packet periodically sent to cloud 101 detects Border Gateway 102 to cloud 101 is unobstructed.Wherein, week is preset
Phase can be configured according to actual demand, such as a few minutes or several seconds.Cloud 101 is used as server, edge
Gateway 102 is equivalent in telling that 101 Border Gateway 102 of cloud is currently running, after sending heartbeat message at this point, cloud 101 can
To feed back heartbeat response message corresponding with heartbeat message, to inform that Border Gateway 102 has received the heartbeat message.In turn,
Border Gateway 102 can be according to whether receiving heartbeat response message judges whether network state is normal.If being appreciated that edge
If network state between gateway 102 and cloud 101 is exception, possible cloud 101 can not receive heartbeat message, it is also possible to edge
Gateway 102 can not receive heartbeat response message, therefore, in this way, can accurately judge between cloud 101
Network state.
Except through the mode of heartbeat message detection, step 202 can also be realized by other embodiments, such as: edge
Gateway 102 can also transmit (MQTT, Message by message queue telemetering after detecting the accessible network of network interface card
Queuing Telemetry Transport) agreement and cloud 101 establishes connection, and MQTT agreement is formed for Internet of Things
A kind of low overhead, low bandwidth occupy instant communication protocol, can be with few code and bandwidth connect remote equipment
Reliable messaging service in real time is provided.101 network state detection ability of cloud, Ke Yishi are contained in MQTT protocol realization frame
When detection and cloud network connection state.The network state with cloud 101 can also be completed by the readjustment of MQTT disconnection
Detection.
Either use which kind of embodiment, Border Gateway 102 can completion network state voluntarily determination, be not required to
Want user or other manually mode is determined.
After executing the step 202, i.e., after the determining network state with cloud 101, if network state is normal condition, that
Border Gateway 102 directly sends the data to cloud 101.If network state is abnormality, step 203 is executed, that is, is obtained
Take the current cache information of the acquisition corresponding spatial cache of equipment 103.Wherein, cache information may include that buffer data size is gentle
Deposit threshold value.Buffer data size represents the size for the data volume being cached with, and it is receptible that cache threshold represents spatial cache institute
Maximum data buffer memory.
After getting current cache information, step 204, for step 204, a kind of optional embodiment party are executed
Formula: whether the buffer data size for judging present collection device 103 is more than cache threshold, is more than cache threshold in buffer data size
When, delete the data cached of the acquisition equipment for meeting default deletion condition；By data buffer storage to the acquisition corresponding caching of equipment 103
In space.
In the embodiment of the present application, before data cached, it is contemplated that, cannot the problem of 102 memory space of Border Gateway
It is immoderate that caching is carried out to disk, acquire whether the corresponding buffer memory of equipment 103 is more than cache threshold by first judging,
If if being more than, just deleting a part of data and caching current data again, gateway memory buffers pressure can be prevented excessive and gone out
Situations such as existing failure, and then guarantee the stability of gateway caches data.
Wherein, for preset deletion condition, can be time for caching in the spatial cache of acquisition equipment 103 is greater than
Preset value it is first data cached.It therefore, may include: the caching sky determined in acquisition equipment 103 when deleting data cached
Between middle caching time be greater than preset value it is first data cached；It deletes formerly data cached.For preset value, can be for example
Cache-time at most first data cached；Or a failure time limit of data, for example, for waterlevel data, it is false
If using hour as period of change, it is little in a hour basic reference value of pervious data, cache-time can be deleted
It has been greater than the data cached of hour.
In the embodiment of the present application, when deleting data cached, the time for deleting caching is greater than the inline cache of preset value
Data can be equivalent to the very long data of cache-time to deleting, this partial data is since the time of caching is longer, data itself
Use value it is little, can avoid deleting important data to a certain extent.
In the embodiment of the present application, when data cached, the sqlite data that can use lightweight are stored, this
Mode possesses high storage efficiency without installing driving.
In the embodiment of the present application, in order to make the corresponding spatial cache of each acquisition equipment 103 on Border Gateway 102, slow
Depositing threshold value etc. is that reasonably, before obtaining cache information (i.e. before execution step 203), to spatial cache or can also delay
Threshold value etc. is deposited to be configured.
As a kind of possible embodiment, the method for determination of cache threshold may is that Border Gateway 102 determines that itself is right
The multiple acquisition equipment 103 answered；The time interval of data is respectively uploaded according to multiple acquisition equipment 103 and uploads the data of data
Amount determines the cache threshold of each acquisition corresponding spatial cache of equipment 103.
Wherein, the corresponding multiple acquisition equipment 103 of Border Gateway 102 can be determined by inquiring to cloud 101；It can also be with
It is determined by the mark of the acquisition equipment 103 stored on Border Gateway 102.It is inquired to cloud 101, it will be understood that on cloud 101
It is stored with Border Gateway 102 and acquires the corresponding relationship of equipment 103, these can be configured and be stored in advance.For adopting
Collect equipment 103 to upload the time interval of data and upload the data volume of data, one is by obtaining each acquisition to cloud 101
The information of equipment 103；Another kind is determined by the information of the acquisition equipment 103 stored on Border Gateway 102.It is equivalent to acquisition
The relevant information of equipment 103 can store beyond the clouds on 101, and on Border Gateway 102 or the two is all stored.
In the embodiment of the present application, the time interval of data is uploaded according to acquisition equipment 103 and uploads the data volume of data
Cache threshold can be more reasonably set, the caching pressure of gateway is reduced.
As a kind of possible embodiment, the method for determination of spatial cache may is that Border Gateway 102 receives cloud
The caching rule of the 101 multiple acquisition equipment 103 corresponding with itself sent；Caching rule is used to indicate each acquisition equipment
The 103 spatial cache method of salary distribution；According to the caching rule distribution corresponding spatial cache of each acquisition equipment 103.
Wherein, caching rule can be is associated with the interval of the transmission data of acquisition equipment 103 and data volume, for example, hair
Send that the interval of data is shorter, data volume is bigger, corresponding spatial cache is bigger；In another example the interval for sending data is longer, data
Measure smaller, corresponding spatial cache is smaller.After determining caching rule, it can be carried out in conjunction with the relevant information of acquisition equipment 103
The distribution of spatial cache.Further, it should be noted that the configuration of spatial cache can also be used as the another of the determination of cache threshold
A step in kind embodiment, i.e., after first determining that spatial cache, spatial cache determine, the corresponding buffer data size of spatial cache
Also it determines simultaneously, then can determine cache threshold according to spatial cache.
It, can also be manually empty from the storage of 101 allocations of edge gateway 102 of cloud by user in addition to this kind of embodiment
Between, Border Gateway 102 is then sent to by cloud 101.
In addition, on Border Gateway 102, cache-time can also be set in addition to allocating cache threshold value and spatial cache, it should
Cache-time can be longest cache-time, be also possible to most short cache-time.It is appreciated that after being cached data,
When network state is restored to be unknown.Therefore, a cache-time can be set, if longest cache-time, then when certain
The cache-time of a data has been more than the longest cache-time, some data is directly deleted, empty to storage to avoid the data
Between occupancy influence the data newly cached.If most short cache-time, then when the cache-time of some data has been more than this
Most short cache-time, no matter whether current network state is restored, and all sends the data to cloud 101, and most short caching is arranged
The data of time may be very important data.
It next referring to figure 3., is the implementing procedure provided by the embodiments of the present application configured to Border Gateway 102
Figure finally makes Border Gateway 102 as shown in figure 3, can carry out the configuration of caching rule, cache-time etc. on 101 beyond the clouds
It is cached according to preparatory configuration, to guarantee the integrality of data.
In the embodiment of the present application, after temporarily being cached data, however, it is determined that restore with the network state in cloud 101
Normally, data are uploaded to cloud 101 again, guarantee that data can also be timely transmitted to cloud while complete.
After Border Gateway 102 executes the step 204, i.e., after data temporary cache having been arrived corresponding memory space, it is
The normal data transmission of guarantee, the data processing method can also include: when determining that network state reverts to normal condition,
Data are sent to cloud according to preset rules by Border Gateway 102.
Wherein it is determined that whether network state reverts to the embodiment phase of normal condition with step 202 in previous embodiment
Together, it is not repeated to introduce herein.
By data according to preset rules be sent to cloud may include: according to the data volume size of data by data in batches
Be sent to the cloud 102.Wherein, every batch of can be understood as slitting number and report, such as a total of 10 data, Ke Yifen
It is primary to send 5 datas to send twice.
It is appreciated that acquisition 103 data of equipment are reported to cloud 101 and used by Border Gateway 102 after network state is restored
Reporting schemes based on number of data, when gateway works offline for a long time, data volume is bigger, if disposably
Upload will generate very big pressure to service bandwidth, and so as to cause the failure of data synchronization, slitting number can be to avoid if reporting
The data disposably uploaded are too many and fail.
It should be noted that when network state is restored, if there is the data of multiple acquisition equipment 103 to be required to be uploaded to cloud
End 101 can determine the data strip that each acquisition 103 every batch of of equipment uploads according to the number of data for needing to upload in total at this time
Number.Such as: need to upload 50 datas of 5 acquisition equipment 103 in total, each acquisition equipment 103 has 10 datas, then
Can control the data uploaded in total in synchronization is 10, then uploading every time for each acquisition equipment 103
Number of data is then 2.
It next referring to figure 4., is the whole implementation flow chart of data processing method provided by the embodiments of the present application, in conjunction with
Previous embodiment, as shown in figure 4, the flow direction of data includes from acquisition equipment 103 to Border Gateway 102 and Border Gateway 102
Local cache processing and Border Gateway 102 arrive cloud 101, and there are also friendships between cloud 101 and Border Gateway 102
Mutually.
Based on the same inventive concept, next referring to figure 5., a kind of data processing equipment is also provided in the embodiment of the present application
300, it is applied to Border Gateway 102, comprising: receiving module 301, processing module 302 and acquisition module 303.
Receiving module 301 is used to receive the data that acquisition equipment 103 is sent, and the data are the number for being sent to cloud
According to.Processing module 302 is for determining the currently network state with the cloud.Determining that the network state is abnormality
When, obtain the current cache information that module 303 is used to obtain the acquisition corresponding spatial cache of equipment 103.Processing module 302 is also used
According to the current cache information by the data buffer storage into the spatial cache.
Optionally, processing module 302 is also used to: whether the buffer data size for judging present collection device 103 is more than described
Cache threshold deletes the acquisition equipment 103 for meeting default deletion condition when the buffer data size is more than the cache threshold
It is data cached；By the data buffer storage into the acquisition corresponding spatial cache of equipment 103.
Optionally, processing module 302 is also used to: determining itself corresponding multiple acquisition equipment 103；It is set according to multiple acquisitions
The standby 103 respective time interval for uploading data and the data volume for uploading data determine that each acquisition equipment 103 is corresponding described slow
Deposit the cache threshold in space.
Optionally, processing module 302 is also used to: determining that the time cached in the spatial cache of acquisition equipment 103 is greater than
Preset value it is first data cached；It deletes described formerly data cached.
Optionally, receiving module 301 is also used to receive multiple acquisition equipment 103 corresponding with itself of the transmission of cloud 101
Caching rule；The caching rule is used to indicate the spatial cache method of salary distribution of each acquisition equipment 103；According to the caching
The rule distribution corresponding spatial cache of each acquisition equipment 103.
Optionally, processing module 302 is also used to: whether detection currently can normally access network；It is being currently able to normally
When accessing network, heartbeat message is sent to cloud 101 according to predetermined period；Receive cloud 101 transmission with the heartbeat
When the corresponding heartbeat response message of message, determine that the network state is normal condition；In the institute for not receiving the transmission of cloud 101
When stating heartbeat response message, determine that the network state is the abnormality.
Optionally, when determining that the network state reverts to normal condition, processing module 302 is also used to the data
Cloud 101 is sent to according to preset rules.
Optionally, processing module 302 is also used to the hair according to the data volume size of the data by the data in batches
Give cloud 101.
Modules in data processing equipment 300 shown in fig. 5 are for realizing data provided by the embodiment of the present application
Each step in processing method, therefore, the embodiment of embodiment and each step for modules correspond,
In order to illustrate the succinct of book, it is not repeated to introduce herein.
Based on the same inventive concept, the embodiment of the present application also provides a kind of readable storage medium storing program for executing, the readable storage medium storing program for executing
On be stored with computer program, the computer program executes at data provided by the embodiment of the present application when being run by computer
Arbitrary steps in reason method.
In embodiment provided herein, it should be understood that disclosed device and method, it can be by others side
Formula is realized.The apparatus embodiments described above are merely exemplary, for example, the division of the unit, only one kind are patrolled
Function division is collected, there may be another division manner in actual implementation, in another example, multiple units or components can combine or can
To be integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutual
Coupling, direct-coupling or communication connection can be INDIRECT COUPLING or communication link by some communication interfaces, device or unit
It connects, can be electrical property, mechanical or other forms.
In addition, unit may or may not be physically separated as illustrated by the separation member, as unit
The component of display may or may not be physical unit, it can and it is in one place, or may be distributed over more
In a network unit.Some or all of unit therein can be selected to realize this embodiment scheme according to the actual needs
Purpose.
Furthermore each functional module in each embodiment of the application can integrate one independent portion of formation together
Point, it is also possible to modules individualism, an independent part can also be integrated to form with two or more modules.
Herein, relational terms such as first and second and the like be used merely to by an entity or operation with it is another
One entity or operation distinguish, and without necessarily requiring or implying between these entities or operation, there are any this reality
Relationship or sequence.
The above description is only an example of the present application, the protection scope being not intended to limit this application, for ability
For the technical staff in domain, various changes and changes are possible in this application.Within the spirit and principles of this application, made
Any modification, equivalent substitution, improvement and etc. should be included within the scope of protection of this application.
Claims (10)
1. a kind of data processing method, which is characterized in that be applied to Border Gateway, comprising:
The data that acquisition equipment is sent are received, the data are to be sent to the data in cloud；
Determine the currently network state with the cloud；
When determining the network state is abnormality, the current cache letter of the corresponding spatial cache of the acquisition equipment is obtained
Breath；
According to the current cache information by the data buffer storage into the spatial cache.
2. the method according to claim 1, wherein the cache information includes buffer data size and caching threshold
Value, according to the current cache information by the data buffer storage into the spatial cache, comprising:
Whether the buffer data size for judging presently described acquisition equipment is more than the cache threshold, is more than in the buffer data size
When the cache threshold, the data cached of the acquisition equipment for meeting default deletion condition is deleted；
By the data buffer storage into the corresponding spatial cache of the acquisition equipment.
3. according to the method described in claim 2, it is characterized in that, obtaining working as the acquisition corresponding spatial cache of equipment
Before preceding cache information, the method also includes:
Determine itself corresponding multiple acquisition equipment；
Each acquisition is determined according to the data volume that multiple acquisition equipment respectively upload the time interval of data and upload data
The cache threshold of the corresponding spatial cache of equipment.
4. according to the method described in claim 2, it is characterized in that, deleting the acquisition equipment for meeting default deletion condition
It is data cached, comprising:
Determine that the time cached in the spatial cache of the acquisition equipment is greater than the first data cached of preset value；
It deletes described formerly data cached.
5. the method according to claim 1, wherein obtaining working as the acquisition corresponding spatial cache of equipment
Before preceding cache information, the method also includes:
Receive the caching rule for multiple acquisition equipment corresponding with itself that the cloud is sent；The caching rule is used for
Indicate the spatial cache method of salary distribution of each acquisition equipment；
Distribute each acquisition equipment corresponding spatial cache according to the caching rule.
6. the method according to claim 1, wherein determining the currently network state with the cloud, comprising:
Whether detection currently can normally access network；
When being currently able to normally access network, heartbeat message is sent to the cloud according to predetermined period；
When receiving the heartbeat response message corresponding with the heartbeat message that the cloud is sent, the network state is determined
For normal condition；
When not receiving the heartbeat response message that the cloud is sent, determine that the network state is the abnormal shape
State.
7. the method according to claim 1, wherein according to the current cache information by the data buffer storage
After into the spatial cache, the method also includes:
When determining that the network state reverts to normal condition, the data are sent to the cloud according to preset rules.
8. the method according to the description of claim 7 is characterized in that the data are sent to the cloud according to preset rules
End, comprising:
According to the data volume size of the data by the data in batches be sent to the cloud.
9. a kind of data processing equipment, which is characterized in that be applied to Border Gateway, described device includes:
Receiving module, the data sent for receiving acquisition equipment, the data are to be sent to the data in cloud；
Processing module, for determining the currently network state with the cloud；
Module is obtained, it is empty for when determining the network state is abnormality, obtaining the corresponding caching of the acquisition equipment
Between current cache information；
The processing module is also used to the data buffer storage according to the current cache information into the spatial cache.
10. a kind of readable storage medium storing program for executing, which is characterized in that be stored with computer program, the meter on the readable storage medium storing program for executing
It executes when calculation machine program is run by computer such as the step in method of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910831499.3A CN110535965A (en) | 2019-09-03 | 2019-09-03 | A kind of data processing method and device, storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910831499.3A CN110535965A (en) | 2019-09-03 | 2019-09-03 | A kind of data processing method and device, storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110535965A true CN110535965A (en) | 2019-12-03 |
Family
ID=68666804
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910831499.3A Pending CN110535965A (en) | 2019-09-03 | 2019-09-03 | A kind of data processing method and device, storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110535965A (en) |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111063180A (en) * | 2019-12-05 | 2020-04-24 | 深圳市新科聚合网络技术有限公司 | Remote meter reading method based on LoRa |
| CN111245891A (en) * | 2019-12-27 | 2020-06-05 | 特瓦特能源科技有限公司 | Real-time data display method and device |
| CN111277557A (en) * | 2019-12-04 | 2020-06-12 | 珠海派诺科技股份有限公司 | Real-time communication method, equipment and storage medium |
| CN111419216A (en) * | 2020-04-09 | 2020-07-17 | 苏州维伟思医疗科技有限公司 | Electrocardio data transmission system and method |
| CN111522656A (en) * | 2020-04-14 | 2020-08-11 | 北京航空航天大学 | Edge calculation data scheduling and distributing method |
| CN111556087A (en) * | 2020-03-06 | 2020-08-18 | 远景智能国际私人投资有限公司 | Data transmission method, device, equipment and readable storage medium |
| CN111591842A (en) * | 2020-05-30 | 2020-08-28 | 陕西泓源特种设备研究院有限公司 | Voice control elevator method and system based on intelligent gateway |
| CN111681346A (en) * | 2020-05-29 | 2020-09-18 | 软通动力信息技术有限公司 | Prevention and control passing processing method, device, equipment and medium |
| CN111792465A (en) * | 2020-06-04 | 2020-10-20 | 青岛海信智慧家居系统股份有限公司 | Elevator control system and method |
| CN111818124A (en) * | 2020-05-29 | 2020-10-23 | 平安科技（深圳）有限公司 | Data storage method, data storage device, electronic equipment and medium |
| CN111865817A (en) * | 2020-06-23 | 2020-10-30 | 烽火通信科技股份有限公司 | Load balancing control method, device and equipment for remote measuring collector and storage medium |
| CN111970346A (en) * | 2020-08-05 | 2020-11-20 | 深圳市中物互联技术发展有限公司 | Internet of things data transmission method and system |
| CN112445827A (en) * | 2020-11-26 | 2021-03-05 | 中孚信息股份有限公司 | Data security processing system, method and device in cloud office environment |
| CN112511456A (en) * | 2020-12-21 | 2021-03-16 | 北京百度网讯科技有限公司 | Flow control method, flow control device, flow control equipment, storage medium and computer program product |
| CN112666877A (en) * | 2021-02-01 | 2021-04-16 | 亚特智物联技术(广东)有限公司 | Control system of Internet of things |
| CN112954029A (en) * | 2021-01-30 | 2021-06-11 | 上海上实龙创智能科技股份有限公司 | Internet of things data acquisition system capable of continuously acquiring data |
| CN113242456A (en) * | 2021-03-26 | 2021-08-10 | 卡莱特云科技股份有限公司 | GPS data reporting method, playing box and regional advertisement delivery system |
| CN114449045A (en) * | 2021-12-29 | 2022-05-06 | 深圳达实智能股份有限公司 | Internet of things edge gateway data caching method and device |
| CN114500669A (en) * | 2021-12-31 | 2022-05-13 | 珠海派诺科技股份有限公司 | Real-time communication method and system based on Internet of things, storage medium and electronic equipment |
| CN114531379A (en) * | 2021-12-30 | 2022-05-24 | 爱普（福建）科技有限公司 | Remote operation and maintenance method and system for edge gateway |
| CN115083034A (en) * | 2022-05-31 | 2022-09-20 | 北京北方华创微电子装备有限公司 | Message reporting method and device of semiconductor equipment and semiconductor equipment |
| CN115243318A (en) * | 2022-07-01 | 2022-10-25 | 华迪计算机集团有限公司 | Internet of things data transparent transmission method and system |
| CN115250293A (en) * | 2022-06-30 | 2022-10-28 | 深圳水趣智能零售系统有限公司 | Data uploading method and device and computer readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106776378A (en) * | 2016-12-05 | 2017-05-31 | 宇龙计算机通信科技(深圳)有限公司 | It is a kind of to clear up data cached method and device |
| CN107666509A (en) * | 2017-08-28 | 2018-02-06 | 深圳市诚壹科技有限公司 | Buffer control method and server |
| CN109743209A (en) * | 2019-01-21 | 2019-05-10 | 北京百度网讯科技有限公司 | Method for transmitting information and device |
| CN109995865A (en) * | 2019-04-03 | 2019-07-09 | 广东工业大学 | A kind of request responding method and device of the data information based on mobile edge calculations |
-
2019
- 2019-09-03 CN CN201910831499.3A patent/CN110535965A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106776378A (en) * | 2016-12-05 | 2017-05-31 | 宇龙计算机通信科技(深圳)有限公司 | It is a kind of to clear up data cached method and device |
| CN107666509A (en) * | 2017-08-28 | 2018-02-06 | 深圳市诚壹科技有限公司 | Buffer control method and server |
| CN109743209A (en) * | 2019-01-21 | 2019-05-10 | 北京百度网讯科技有限公司 | Method for transmitting information and device |
| CN109995865A (en) * | 2019-04-03 | 2019-07-09 | 广东工业大学 | A kind of request responding method and device of the data information based on mobile edge calculations |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111277557A (en) * | 2019-12-04 | 2020-06-12 | 珠海派诺科技股份有限公司 | Real-time communication method, equipment and storage medium |
| CN111277557B (en) * | 2019-12-04 | 2023-01-03 | 珠海派诺科技股份有限公司 | Real-time communication method, equipment and storage medium |
| CN111063180A (en) * | 2019-12-05 | 2020-04-24 | 深圳市新科聚合网络技术有限公司 | Remote meter reading method based on LoRa |
| CN111245891A (en) * | 2019-12-27 | 2020-06-05 | 特瓦特能源科技有限公司 | Real-time data display method and device |
| CN111556087A (en) * | 2020-03-06 | 2020-08-18 | 远景智能国际私人投资有限公司 | Data transmission method, device, equipment and readable storage medium |
| CN111556087B (en) * | 2020-03-06 | 2023-02-28 | 远景智能国际私人投资有限公司 | Data transmission method, device, equipment and readable storage medium |
| CN111419216A (en) * | 2020-04-09 | 2020-07-17 | 苏州维伟思医疗科技有限公司 | Electrocardio data transmission system and method |
| CN111522656A (en) * | 2020-04-14 | 2020-08-11 | 北京航空航天大学 | Edge calculation data scheduling and distributing method |
| CN111818124A (en) * | 2020-05-29 | 2020-10-23 | 平安科技（深圳）有限公司 | Data storage method, data storage device, electronic equipment and medium |
| CN111681346A (en) * | 2020-05-29 | 2020-09-18 | 软通动力信息技术有限公司 | Prevention and control passing processing method, device, equipment and medium |
| WO2021120585A1 (en) * | 2020-05-29 | 2021-06-24 | 平安科技（深圳）有限公司 | Data storage method, apparatus, device and storage medium |
| CN111591842A (en) * | 2020-05-30 | 2020-08-28 | 陕西泓源特种设备研究院有限公司 | Voice control elevator method and system based on intelligent gateway |
| CN111792465A (en) * | 2020-06-04 | 2020-10-20 | 青岛海信智慧家居系统股份有限公司 | Elevator control system and method |
| CN111792465B (en) * | 2020-06-04 | 2022-05-20 | 青岛海信智慧家居系统股份有限公司 | Elevator control system and method |
| CN111865817A (en) * | 2020-06-23 | 2020-10-30 | 烽火通信科技股份有限公司 | Load balancing control method, device and equipment for remote measuring collector and storage medium |
| CN111970346A (en) * | 2020-08-05 | 2020-11-20 | 深圳市中物互联技术发展有限公司 | Internet of things data transmission method and system |
| CN112445827A (en) * | 2020-11-26 | 2021-03-05 | 中孚信息股份有限公司 | Data security processing system, method and device in cloud office environment |
| CN112511456A (en) * | 2020-12-21 | 2021-03-16 | 北京百度网讯科技有限公司 | Flow control method, flow control device, flow control equipment, storage medium and computer program product |
| CN112511456B (en) * | 2020-12-21 | 2024-03-22 | 北京百度网讯科技有限公司 | Flow control method, apparatus, device, storage medium, and computer program product |
| CN112954029A (en) * | 2021-01-30 | 2021-06-11 | 上海上实龙创智能科技股份有限公司 | Internet of things data acquisition system capable of continuously acquiring data |
| CN112666877A (en) * | 2021-02-01 | 2021-04-16 | 亚特智物联技术(广东)有限公司 | Control system of Internet of things |
| CN113242456B (en) * | 2021-03-26 | 2023-05-09 | 卡莱特云科技股份有限公司 | GPS data reporting method, playing box and regional advertisement putting system |
| CN113242456A (en) * | 2021-03-26 | 2021-08-10 | 卡莱特云科技股份有限公司 | GPS data reporting method, playing box and regional advertisement delivery system |
| CN114449045A (en) * | 2021-12-29 | 2022-05-06 | 深圳达实智能股份有限公司 | Internet of things edge gateway data caching method and device |
| CN114531379A (en) * | 2021-12-30 | 2022-05-24 | 爱普（福建）科技有限公司 | Remote operation and maintenance method and system for edge gateway |
| CN114500669A (en) * | 2021-12-31 | 2022-05-13 | 珠海派诺科技股份有限公司 | Real-time communication method and system based on Internet of things, storage medium and electronic equipment |
| CN115083034A (en) * | 2022-05-31 | 2022-09-20 | 北京北方华创微电子装备有限公司 | Message reporting method and device of semiconductor equipment and semiconductor equipment |
| CN115250293A (en) * | 2022-06-30 | 2022-10-28 | 深圳水趣智能零售系统有限公司 | Data uploading method and device and computer readable storage medium |
| CN115243318A (en) * | 2022-07-01 | 2022-10-25 | 华迪计算机集团有限公司 | Internet of things data transparent transmission method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110535965A (en) | A kind of data processing method and device, storage medium | |
| US11102123B2 (en) | Sensor network system | |
| CN107390650B (en) | A kind of data collection system based on Internet of Things and the data compression method based on the system | |
| KR20220083851A (en) | Method and apparatus for transmitting data in Internet of Things system, gateway device and storage medium thereof | |
| CN107395572B (en) | Data processing method and Internet of things gateway | |
| CN111611129B (en) | Performance monitoring method and device of PaaS cloud platform | |
| CN104023083A (en) | Method and device for log collection cluster load balance | |
| CN106453541A (en) | Data synchronization method, server and data synchronization system | |
| CN112887217A (en) | Control data packet sending method, model training method, device and system | |
| CN108614450A (en) | Mechanical system and its method under a kind of Internet of Things | |
| CN111064788B (en) | Signal transmission method, robot, and computer-readable storage medium | |
| CN108512634A (en) | A kind of method and relevant device of data processing | |
| US20220247651A1 (en) | System and method for network and computation performance probing for edge computing | |
| CN111405531A (en) | Method, medium, terminal and device for improving communication quality | |
| CN115002238A (en) | Traffic transmission method, device, equipment and storage medium | |
| CN106357478B (en) | A kind of server cluster monitoring method and system | |
| CN104243473B (en) | A kind of method and device of data transmission | |
| JP2003101518A (en) | Live mobile camera system | |
| CN109347985A (en) | A kind of multi-service processing method based on edge calculations, device and Edge Server | |
| CN110290163A (en) | A kind of data processing method and device | |
| CN108632136A (en) | Patrol river data processing method and device | |
| CN109271454A (en) | A kind of method and the network equipment that data are synchronous | |
| JP5271737B2 (en) | Data collection system and transmission control device | |
| CN107071038A (en) | A kind of method and device of transmitting multimedia data | |
| CN112653717B (en) | Multi-cloud cooperation distributed system and application distribution method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191203 |
|
| RJ01 | Rejection of invention patent application after publication |