CN110113413A - A kind of method of data processing in Internet of Things - Google Patents
A kind of method of data processing in Internet of Things Download PDFInfo
- Publication number
- CN110113413A CN110113413A CN201910366801.2A CN201910366801A CN110113413A CN 110113413 A CN110113413 A CN 110113413A CN 201910366801 A CN201910366801 A CN 201910366801A CN 110113413 A CN110113413 A CN 110113413A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- reading
- nodes
- assigned
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Computer Security & Cryptography (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention provides a kind of methods of data processing in Internet of Things, by carrying out fusion duplicate removal to acquisition data, compression processing is carried out to initial data using the data compression method of lightweight, the data of the data and a upper preservation that more currently read and the value calculating difference of the degree of compression, the difference is greater than preset threshold value and is then put into inventory list, otherwise the data currently read are assigned to the data of a upper preservation, and the method handled by task queue, calculate node expense selects optimum way, it is big to solve existing Internet of things node data processing amount, broadband and the larger technical problem of energy consumption.
Description
Technical field
The present invention relates to technical field of data processing, in particular to the method for data processing in a kind of Internet of Things.
Background technique
With the fast development of information technology, the trend of explosive growth is also presented in the data that various applications generate, this gives
The storage and backup of data cause certain challenge.With the extensive portion of magnanimity sensor, intelligent terminal, network communication equipment
Administration, a globalization Internet of Things being made of terminal device gradually form.Currently, an obstacle facing of Internet of Things development be for
User provides the search service based on sensor information, and the information needed for helping user to obtain it is simultaneously used.In Internet of Things
Node and sensor all produce a large amount of data all the time, the quantity of these data is far longer than in traditional internet
The data volume that oneself has.On the one hand, the object searched in traditional internet is the unstructured number such as webpage, text, audio, video
According to these data are static, sections that is non real-time, and acquiring in Internet of Things search field, the object of search for sensor
Three-point state information, these information are structurings, and are highly dynamic variations.
Therefore, existing data processing method is not suitable for the high speed processing of big data, and the communication overhead of system is very big,
It can not be suitable for large-scale application, node data is increasing, communication bring broadband and energy between gateway and node
Amount consumption is very huge.It would therefore be highly desirable to the method for proposing data processing in a kind of Internet of Things.
Summary of the invention
The embodiment of the invention provides a kind of methods of data processing in Internet of Things, by carrying out fusion to acquisition data
Weight carries out compression processing, and the method handled by task queue to initial data using the data compression method of lightweight,
Calculate node expense selects optimum way, to solve big existing Internet of things node data processing amount, broadband and energy consumption
Larger technical problem.
To solve the above-mentioned problems, the invention discloses following technical solutions:
A kind of method of data processing in Internet of Things is provided, which comprises
Step 1: sensor acquisition point data is first merged duplicate removal again, multiple tasks queue, each task queue tool are established
There are multiple nodes, including non-leaf nodes and leaf node；
Walk upper and lower slope and the permission of respectively system that poly- two, each task queue initializes an acquisition point data
Maximum value and minimum value, initialize current collection point slope be 0；The slope is certain a period of time of collection point in system
The data volume difference at interval and the ratio of time interval；
Step 3: reading first collection point is assigned to the data an of reading and the data of a upper preservation, by institute
The data for stating a reading are added to inventory list；
Step 4: judging whether collection point number is more than or equal to 2, if meeting condition executes step 5, step is otherwise executed
Rapid ten；
Step 5: the collection point of reading to be assigned to the data currently read, according to data currently read, described
The data of a upper preservation and the value calculating difference of the degree of compression, and judge whether the threshold value less than setting, if it is less than the threshold
Value thens follow the steps six, otherwise executes step 7；
Step 6: the data currently read to be assigned to the data of a upper reading, also whether judgement current
There is not processed data point, execution step 5 is recycled if there is also not processed data point, otherwise executes step 10；
Step 7: the data of a upper reading to be assigned to the data of a upper preservation, by described upper one
The data of reading are added to inventory list；
Step 8: collection point to be assigned to the data currently read, according to data currently read, upper one described
The data of preservation and the value of the degree of compression calculate and update the upper and lower slope；
Step 9: judgement is current, whether there are also not processed data points, if there is also if not processed data point
Circulation executes step 10, otherwise executes step 12；
Step 10: executing step if current slope is more than or equal to lower slope or current slope is less than or equal to upper slope
Five, otherwise execute step 11；
Step 11: the data of a upper reading are added to inventory list, by the data of a upper reading
The data of a upper preservation are assigned to, upper and lower slope is updated, the data currently read are assigned to a upper reading
Data；
Step 12: the data of a upper reading are added to inventory list, compression process terminates.
Further, the task queue execution method wherein in step 1 includes: as follows
1) judge whether task queue is empty.If task queue is sky, algorithm terminates, and otherwise executes the 2) step；
2) all nodes calculate the expected time E that oneself executes next task；
3) bandwidth of all non-leaf nodes detection each and oneself child node phase connecting lines in setting；
4) all non-leaf nodes obtain the expected time E of child node to its all child nodes sending instruction in tree；
5) leaf node is connected to after instruction itself expected time E returning to father node；
6) since the non-leaf nodes of the bottom, when it receives the feedback result of all child nodes, by result set together with
The bandwidth of the transmission line of oneself and all child nodes returns to its father node together；
7) 6) step of recursive call, until host node receives the result set that all childs are returned；
8) host node calculates the coefficient C of all nodes；
9) task is taken out from task queue, is assigned the task to the smallest node of limit coefficient, is then branched to
1) step.
Further, the coefficient C that wherein host node calculates all nodes includes: to set S node to calculate+1 list of kth
Coefficient C when position/tasks(k+1)；
ThenWherein a and b is the weight of E and T respectively, and n is total node number amount,
E is the expected time that node s handles some unit task, and T is to save+1 unit task of kth from A node-node transmission to B
The predicted transmission time of point.
The embodiment of the invention provides a kind of methods of data processing in Internet of Things, by carrying out fusion to acquisition data
Weight carries out compression processing, data more currently read and one upper to initial data using the data compression method of lightweight
The data of preservation and the value calculating difference of the degree of compression, the difference are greater than preset threshold value and are then put into inventory list, otherwise will be current
The data of reading are assigned to the data of a preservation, and the method handled by task queue, the selection of calculate node expense
Optimum way, to solve the technical problem that existing Internet of things node data processing amount is big, broadband and energy consumption are larger.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is the present invention
Some embodiments for those of ordinary skill in the art without creative efforts, can also basis
These attached drawings obtain other attached drawings.
Fig. 1 is the flow diagram of the method for data processing in Internet of Things in one embodiment of the invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments, based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
Referring to Fig. 1, one embodiment of the invention proposes the flow diagram of a kind of method of data processing in Internet of Things,
It comprises the following steps that
Step 1: sensor acquisition point data is first merged duplicate removal again, multiple tasks queue, each task queue tool are established
There are multiple nodes, including non-leaf nodes and leaf node；
Walk upper and lower slope and the permission of respectively system that poly- two, each task queue initializes an acquisition point data
Maximum value and minimum value, initialize current collection point slope be 0；The slope is certain a period of time of collection point in system
The data volume difference at interval and the ratio of time interval；
Step 3: reading first collection point is assigned to the data an of reading and the data of a upper preservation, by institute
The data for stating a reading are added to inventory list；
Step 4: judging whether collection point number is more than or equal to 2, if meeting condition executes step 5, step is otherwise executed
Rapid ten；
Step 5: the collection point of reading to be assigned to the data currently read, according to data currently read, described
The data of a upper preservation and the value calculating difference of the degree of compression, and judge whether the threshold value less than setting, if it is less than the threshold
Value thens follow the steps six, otherwise executes step 7；
Step 6: the data currently read to be assigned to the data of a upper reading, also whether judgement current
There is not processed data point, execution step 5 is recycled if there is also not processed data point, otherwise executes step 10；
Step 7: the data of a upper reading to be assigned to the data of a upper preservation, by described upper one
The data of reading are added to inventory list；
Step 8: collection point to be assigned to the data currently read, according to data currently read, upper one described
The data of preservation and the value of the degree of compression calculate and update the upper and lower slope；
Step 9: judgement is current, whether there are also not processed data points, if there is also if not processed data point
Circulation executes step 10, otherwise executes step 12；
Step 10: executing step if current slope is more than or equal to lower slope or current slope is less than or equal to upper slope
Five, otherwise execute step 11；
Step 11: the data of a upper reading are added to inventory list, by the data of a upper reading
The data of a upper preservation are assigned to, upper and lower slope is updated, the data currently read are assigned to a upper reading
Data；
Step 12: the data of a upper reading are added to inventory list, compression process terminates.
And wherein the task queue execution method in step 1 includes: as follows
1) judge whether task queue is empty.If task queue is sky, algorithm terminates, and otherwise executes the 2) step；
2) all nodes calculate the expected time E that oneself executes next task；
3) bandwidth of all non-leaf nodes detection each and oneself child node phase connecting lines in setting；
4) all non-leaf nodes obtain the expected time E of child node to its all child nodes sending instruction in tree；
5) leaf node is connected to after instruction itself expected time E returning to father node；
6) since the non-leaf nodes of the bottom, when it receives the feedback result of all child nodes, by result set together with
The bandwidth of the transmission line of oneself and all child nodes returns to its father node together；
7) 6) step of recursive call, until host node receives the result set that all childs are returned；
8) host node calculates the coefficient C of all nodes；
9) task is taken out from task queue, is assigned the task to the smallest node of limit coefficient, is then branched to
1) step.
And the coefficient C that host node calculates all nodes includes: when setting S node to calculate+1 unit task of kth
Coefficient Cs(k+1)；
ThenWherein a and b is the weight of E and T respectively, and n is total node number amount,
E is the expected time that node s handles some unit task, and T is to save+1 unit task of kth from A node-node transmission to B
The predicted transmission time of point.
The embodiment of the invention provides a kind of methods of data processing in Internet of Things, by carrying out fusion to acquisition data
Weight carries out compression processing, data more currently read and one upper to initial data using the data compression method of lightweight
The data of preservation and the value calculating difference of the degree of compression, the difference are greater than preset threshold value and are then put into inventory list, otherwise will be current
The data of reading are assigned to the data of a preservation, and the method handled by task queue, the selection of calculate node expense
Optimum way, to solve the technical problem that existing Internet of things node data processing amount is big, broadband and energy consumption are larger.
For convenience of description, each section of apparatus above is divided into various modules with function or unit describes respectively.Certainly,
Each module or the function of unit can be realized in same or multiple softwares or hardware in carrying out the present invention.
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the present invention, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic
Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the scope of the invention.
Claims (3)
1. a kind of method of data processing in Internet of Things, which is characterized in that the described method includes:
Step 1: sensor acquisition point data is first merged duplicate removal again, multiple tasks queue, each task queue tool are established
There are multiple nodes, including non-leaf nodes and leaf node；
Walk upper and lower slope and the permission of respectively system that poly- two, each task queue initializes an acquisition point data
Maximum value and minimum value, initialize current collection point slope be 0；The slope is certain a period of time of collection point in system
The data volume difference at interval and the ratio of time interval；
Step 3: reading first collection point is assigned to the data an of reading and the data of a upper preservation, on described
The data of one reading are added to inventory list；
Step 4: judging whether the collection point number is more than or equal to 2, if meeting condition executes step 5, step is otherwise executed
Rapid ten；
Step 5: the collection point of reading to be assigned to the data currently read, according to the data currently read, described upper one
The data of a preservation and the value calculating difference of the degree of compression, and judge whether the threshold value less than setting, then if it is less than the threshold value
Step 6 is executed, step 7 is otherwise executed；
Step 6: the data currently read to be assigned to the data of a upper reading, judgement is current, and whether there are also not
Processed data point recycles execution step 5 if there is also not processed data point, otherwise executes step 10；
Step 7: the data of a upper reading to be assigned to the data of a upper preservation, by a upper reading
Data be added to inventory list；
Step 8: collection point is assigned to the data currently read, according to data currently read, upper one described
The data of preservation and the value calculating difference of the degree of compression, and update the upper and lower slope；
Step 9: judgement is current, whether there are also not processed data points, recycle if there is also not processed data point
Step 10 is executed, step 12 is otherwise executed；
Step 10: if current slope is more than or equal to the lower slope or the current slope is less than or equal to the upper slope,
Step 5 is executed, step 11 is otherwise executed；
Step 11: the data of a upper reading are added to the inventory list, by the data of a upper reading
The data of a upper preservation are assigned to, the upper and lower slope is updated, the data currently read is assigned on described
The data of one reading；
Step 12: the data of a upper reading are added to the inventory list, compression process terminates.
2. wherein the task queue in step 1 executes the method according to claim 1, wherein further
Method includes: as follows
1) judge whether task queue is empty, if the task queue is sky, algorithm terminates, otherwise execute the 2) step；
2) all nodes calculate the expected time E that oneself executes next task；
3) bandwidth of all non-leaf nodes the detection each and oneself child node phase connecting lines in setting；
4) all non-leaf nodes obtain the expected time E of child node to its all child nodes sending instruction in tree；
5) leaf node is connected to after instruction itself expected time E returning to father node；
6) since the non-leaf nodes of the bottom, when it receives the feedback result of all child nodes, by result set
Its father node is returned to together together with the bandwidth of the transmission line of oneself and all child nodes；
7) 6) step of recursive call, until host node receives the result set that all childs are returned；
8) host node calculates the coefficient C of all nodes；
9) task is taken out from task queue, assigns the task to the smallest node of limit coefficient, then branches to the 1)
Step.
3. according to the method described in claim 2, wherein host node calculates all nodes it is characterized in that, further
Coefficient C includes: the coefficient Cs (k+1) set when S node calculates+1 unit task of kth；
Then Cs (k+1)=aE_s^n (k+1)+bT_ (n- > s) (k+1)；Wherein a and b is the weight of E and T respectively, and n is total
Number of nodes, E are that node s handles expected time of some unit task, and T is by+1 unit task of kth from A node
It is transferred to the predicted transmission time of B node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910366801.2A CN110113413A (en) | 2019-04-30 | 2019-04-30 | A kind of method of data processing in Internet of Things |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910366801.2A CN110113413A (en) | 2019-04-30 | 2019-04-30 | A kind of method of data processing in Internet of Things |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110113413A true CN110113413A (en) | 2019-08-09 |
Family
ID=67488183
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910366801.2A Pending CN110113413A (en) | 2019-04-30 | 2019-04-30 | A kind of method of data processing in Internet of Things |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110113413A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111417096A (en) * | 2019-12-31 | 2020-07-14 | 咻享智能(深圳)有限公司 | Wireless Internet of things node management method and related device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103327562A (en) * | 2013-06-28 | 2013-09-25 | 南京邮电大学 | Method for selecting wireless multi-media sensor network nodes based on related space mapping |
| CN103781116A (en) * | 2013-11-01 | 2014-05-07 | 上海交通大学 | Data integration method for wireless sensor network based on distributed storage |
| CN105007599A (en) * | 2015-07-01 | 2015-10-28 | 湘潭大学 | Data compression collection method based on connectivity clustering |
| CN105072194A (en) * | 2015-08-27 | 2015-11-18 | 南京大学 | Structure and method for recovering stored data in distributed file system |
| KR20170068874A (en) * | 2015-12-10 | 2017-06-20 | 주식회사 리드앤 | Method of compression of data for transmission of vital information |
-
2019
- 2019-04-30 CN CN201910366801.2A patent/CN110113413A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103327562A (en) * | 2013-06-28 | 2013-09-25 | 南京邮电大学 | Method for selecting wireless multi-media sensor network nodes based on related space mapping |
| CN103781116A (en) * | 2013-11-01 | 2014-05-07 | 上海交通大学 | Data integration method for wireless sensor network based on distributed storage |
| CN105007599A (en) * | 2015-07-01 | 2015-10-28 | 湘潭大学 | Data compression collection method based on connectivity clustering |
| CN105072194A (en) * | 2015-08-27 | 2015-11-18 | 南京大学 | Structure and method for recovering stored data in distributed file system |
| KR20170068874A (en) * | 2015-12-10 | 2017-06-20 | 주식회사 리드앤 | Method of compression of data for transmission of vital information |
Non-Patent Citations (1)
| Title |
|---|
| 崔征: ""基于压缩感知的无线传感器网络数据压缩算法的研究"", 《中国优秀硕士学位论文全文数据库（电子期刊）》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111417096A (en) * | 2019-12-31 | 2020-07-14 | 咻享智能(深圳)有限公司 | Wireless Internet of things node management method and related device |
| CN111417096B (en) * | 2019-12-31 | 2021-10-22 | 咻享智能(深圳)有限公司 | Wireless Internet of things node management method and related device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6755325B2 (en) | State control method and equipment | |
| CN110428046B (en) | Method and device for acquiring neural network structure and storage medium | |
| US20180307787A1 (en) | Accelerating particle-swarm algorithms | |
| US20130173511A1 (en) | Using global and local catastrophes across sub-populations in parallel evolutionary computing | |
| CN111461345A (en) | Deep learning model training method and device | |
| CN108089918B (en) | Graph computation load balancing method for heterogeneous server structure | |
| CN114202027A (en) | Execution configuration information generation method, model training method and device | |
| CN116414559A (en) | Method for modeling and distributing unified computing power identification, storage medium and electronic equipment | |
| CN116166405A (en) | Neural network task scheduling strategy determination method and device in heterogeneous scene | |
| CN113627536A (en) | Model training method, video classification method, device, equipment and storage medium | |
| CN110113413A (en) | A kind of method of data processing in Internet of Things | |
| CN108287859B (en) | Multimedia information retrieval method and device | |
| CN106933654A (en) | A kind of virtual machine based on caching starts method | |
| CN115878860A (en) | Menu generation method, device, server equipment and medium | |
| CN112256983B (en) | Navigation information processing method and device, electronic equipment and storage medium | |
| CN112738225B (en) | Edge calculation method based on artificial intelligence | |
| CN114237824A (en) | Fault positioning method and device, computer readable medium and electronic equipment | |
| CN112580803B (en) | Model acquisition method, apparatus, electronic device, storage medium, and program product | |
| CN112749540A (en) | Text matching method, training method, device and equipment of text matching model | |
| CN109190003A (en) | For determining the method and apparatus of list page node | |
| CN104410537A (en) | Generation system of tree type network topological graph and generation method of tree type network topological graph | |
| CN116028233B (en) | Digital object organization and sharing method and device of AI computing resource | |
| CN109344953B (en) | Cloud service combination method | |
| CN115841197A (en) | Path planning method, device, equipment and storage medium | |
| CN105827418A (en) | Communication network alarm correlation method and communication network alarm correlation device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190809 |