CN112332981B - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN112332981B CN112332981B CN202011291328.5A CN202011291328A CN112332981B CN 112332981 B CN112332981 B CN 112332981B CN 202011291328 A CN202011291328 A CN 202011291328A CN 112332981 B CN112332981 B CN 112332981B
- Authority
- CN
- China
- Prior art keywords
- data
- internet
- things
- gateway
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/08—Key distribution or management, e.g. generation, sharing or updating, of cryptographic keys or passwords
- H04L9/0861—Generation of secret information including derivation or calculation of cryptographic keys or passwords
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/08—Key distribution or management, e.g. generation, sharing or updating, of cryptographic keys or passwords
- H04L9/088—Usage controlling of secret information, e.g. techniques for restricting cryptographic keys to pre-authorized uses, different access levels, validity of crypto-period, different key- or password length, or different strong and weak cryptographic algorithms
Abstract
The embodiment of the invention provides a data processing method and device, wherein the method comprises the following steps: the method comprises the steps that an Internet of things gateway obtains first data collected by Internet of things equipment; the first data are encrypted data of the Internet of things equipment; the gateway of the Internet of things obtains the equipment identifier of the equipment of the Internet of things, and decrypts the first data according to the equipment identifier to obtain second data; the gateway of the Internet of things filters the second data according to a preset filtering rule to obtain third data; the gateway of the Internet of things determines a data type corresponding to the third data and determines correction information and compression information corresponding to the data type; the gateway of the Internet of things corrects the third data according to the correction information to obtain fourth data; and the gateway of the Internet of things compresses the fourth data according to the compressed information to obtain target data. By the embodiment of the invention, the data is preprocessed before the data transmission, the bandwidth occupied by the data transmission is reduced, and the pressure of the server for processing the data is relieved.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a method and apparatus for data processing.
Background
At present, the development and application of big data have a plurality of problems at the same time of rapid development, for example: insufficient open sharing of data, insufficient application field, insufficient development and utilization of data resources, serious unordered abuse phenomenon and the like.
In the construction of smart cities, big data plays an important role, and according to the experiences of the construction of the smart cities in the past, due to the lack of a unified development management platform, each city application can only be independently constructed, so that the problems of data barriers and application barriers exist among each city application, a large amount of information islands are formed, and the effect and the value of the big data cannot be exerted. Therefore, a unified cloud platform capable of being popularized and used is needed to be explored, so that the stripe segmentation among urban applications is broken, the information gap is eliminated, and mass fusion of big data is realized.
In the process of constructing the unified cloud platform, an area Internet of things platform is established for management equipment and equipment data, and in the area Internet of things platform, a large amount of equipment data acquired by Internet of things equipment in real time contains invalid and incorrect data, and if the data directly flow into the area Internet of things platform through a server, a large amount of network bandwidth is occupied, and the pressure of the server for processing the data is increased.
Disclosure of Invention
In view of the above, a method and apparatus for data processing is proposed to overcome or at least partially solve the above problems, comprising:
the data processing method is applied to an area internet of things platform, and the area internet of things platform is provided with an internet of things gateway, and the method comprises the following steps:
the gateway of the Internet of things acquires first data acquired by equipment of the Internet of things; the first data are encrypted data of the internet of things equipment;
the gateway of the Internet of things acquires the equipment identifier of the equipment of the Internet of things, and decrypts the first data according to the equipment identifier to obtain second data;
the gateway of the Internet of things filters the second data according to preset filtering rules to obtain third data;
the gateway of the Internet of things determines a data type corresponding to the third data and determines correction information and compression information corresponding to the data type;
the gateway of the Internet of things corrects the third data according to the correction information to obtain fourth data;
and the gateway of the Internet of things compresses the fourth data according to the compressed information to obtain target data.
Optionally, the gateway of the internet of things compresses the fourth data according to the compression information to obtain target data, including:
acquiring the data volume of the fourth data;
judging whether the data volume is larger than a preset data volume or not;
and when the data volume is larger than the preset data volume, compressing the fourth data according to the compression information to obtain target data.
Optionally, the method further comprises:
determining encryption information corresponding to the data type;
and carrying out encryption processing on the target data according to the encryption information.
Optionally, the correction information includes correction format information, and the correcting the third data according to the correction information to obtain fourth data includes:
and carrying out format conversion processing on the third data according to the corrected format information to obtain fourth data.
Optionally, the data type includes a character type and a non-character type, and the determining, by the gateway of the internet of things, the data type corresponding to the third data and determining the correction information corresponding to the data type includes:
determining a data type corresponding to the third data;
judging whether the data type is non-character type;
and when the data type is a non-character type, determining that the correction format information corresponding to the data type is a character type.
Optionally, the regional internet of things platform is deployed with a cloud server, and further includes:
and sending the target data to the cloud server.
Optionally, the compression information includes compression format information, compression ratio information, and compression speed information.
A data processing apparatus applied to an area internet of things platform, the area internet of things platform being deployed with an internet of things gateway, the apparatus comprising:
the first data acquisition module is used for acquiring first data acquired by the Internet of things equipment by the Internet of things gateway; the first data are encrypted data of the internet of things equipment;
the first data decryption module is used for acquiring the equipment identifier of the equipment of the Internet of things by the gateway of the Internet of things, decrypting the first data according to the equipment identifier, and obtaining second data;
the second data filtering module is used for filtering the second data according to preset filtering rules by the gateway of the Internet of things to obtain third data;
the third data processing module is used for determining a data type corresponding to the third data and determining correction information and compression information corresponding to the data type;
the third data correction module is used for correcting the third data according to the correction information by the gateway of the Internet of things to obtain fourth data;
and the fourth data compression module is used for compressing the fourth data according to the compression information by the Internet of things gateway to obtain target data.
A server comprising a processor, a memory and a computer program stored on the memory and capable of running on the processor, which when executed by the processor implements a method of data processing as described above.
A computer readable storage medium having stored thereon a computer program which when executed by a processor implements a method of data processing as described above.
The embodiment of the invention has the following advantages:
according to the embodiment of the invention, the first data acquired by the Internet of things equipment is acquired through the Internet of things gateway; the method comprises the steps that first data are encrypted data of Internet of things equipment, an Internet of things gateway obtains equipment identification of the Internet of things equipment, decrypts the first data according to the equipment identification to obtain second data, the Internet of things gateway filters the second data according to preset filtering rules to obtain third data, the Internet of things gateway determines data types corresponding to the third data and determines correction information and compression information corresponding to the data types, the Internet of things gateway corrects the third data according to the correction information to obtain fourth data, and the Internet of things gateway compresses the fourth data according to the compression information to obtain target data, so that data transmission occupied bandwidth can be reduced, and cloud server processing data pressure can be reduced.
Drawings
In order to more clearly illustrate the technical solutions of the present invention, the drawings that are needed in the description of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.
FIG. 1 is a diagram of an overall architecture of a cloud platform according to an embodiment of the present invention;
FIG. 2 is a deployment diagram of a platform of regional Internet of things according to an embodiment of the present invention;
FIG. 3 is a flow chart of steps of a method for data processing according to an embodiment of the present invention;
FIG. 4 is a flow chart of steps of another method for data processing according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the construction of the smart city, a standard unified, an entrance unified, a collection unified, a management unified, a service unified and a data unified cross-domain multi-dimensional big data public service cloud platform is built, a city-level unified data standard is built, a data barrier is broken, the Internet of things (Internet of Things, IOT) and system data resources in a convergence area are collected, all business systems of the smart city are loaded, and the ecology of the smart city is created through data open sharing and platform capability open.
The construction targets of the cross-domain multidimensional big data public service cloud platform are that various main bodies, business cooperative mechanisms at all levels and intelligent application in various fields of a smart city are constructed through introducing advanced technologies such as cloud computing, big data, internet of things and mobile interconnection to form an open, interconnected and intelligent smart city ecological system, data sharing in various fields of city management, social folk, resource environment and economic industry is promoted, administrative efficiency, city management capability and resident life quality are improved, industry fusion development is promoted, industry transformation upgrading and business innovation modes are promoted, and popularization and application of the cross-domain multidimensional big data public service cloud platform are realized.
The utility model provides a public service cloud platform of multi-dimensional big data of cross-domain mainly relates to leading-edge IT information technology such as cloud computing, big data, thing networking, artificial intelligence:
1. cloud computing technology: the cloud computing is mainly composed of six core components of elastic computing, network, storage, database, security and middleware, and provides elastic, quick, stable and safe resources and computing power service.
2. Big data technology: the data construction and management are used as cores, and the capabilities of data opening, data integration, data management, data sharing and the like are provided through relevant components such as data calculation, data development, data analysis, data visualization and the like.
3. The technology of the Internet of things: the internet of things platform provides services such as one-stop type equipment access, equipment management, monitoring operation and maintenance, safety guarantee and the like, and can provide an internet of things basic capability support as an important component of a space-time internet of things engine, so that the requirements of intelligent management of novel smart cities in the future are met.
4. Artificial intelligence technology: an AI algorithm development platform is used as a core, and a series of intelligent services are provided through related components such as visual AI, text-to-speech recognition, natural language processing (Natural Language Processing, NLP) platform, map service and the like.
As shown in fig. 1, an internet engine, a space-time internet of things engine, a cross-domain multidimensional big data engine, a regional internet of things sensing system, a fusion service sharing center and a fusion data creation center, a regional application portal, a safe operation and maintenance system, an open operation system and other structures are deployed in the cloud platform.
The following describes each part of the cloud platform in detail:
space-time Internet of things engine
The space-time Internet of things engine is composed of a geographic information system (Geographic Information System or Geo-Information system, GIS), a building information model (Building Information Modeling, BIM) and a regional Internet of things platform and is used for applying the space data and the three-dimensional model to the regional Internet of things.
The geographic information system is a special and very important spatial information system, and can collect, store, manage, calculate, analyze, display and describe the related geographic distribution data in the whole or partial space under the support of a computer hard and software system.
The building information model is an engineering data model based on three-dimensional digital technology and integrating various related information of a building engineering project, and the built model is in continuous deepening and changing along with project progress.
The regional Internet of things platform provides Internet of things foundation capacity support, and can meet the requirement of future urban management. The regional Internet of things platform comprises a sensing equipment management center, a sensing data opening center and a regional event operation center, wherein regional events are constructed by using a large amount of equipment data through the management of equipment and equipment data, and valuable data information is provided for users through the analysis processing of the regional events.
(II) Internet Engine
Cloud efficiency (DevOps) and distributed middleware are deployed in the internet engine, and are used for realizing efficient resource sharing and efficient function sharing of data.
The DevOps is a combination word of Development and Operations, and is a generic term of a set of processes, methods and systems, and is used for promoting Development of application programs/software engineering, technical operation and communication, collaboration and integration between quality assurance departments.
The distributed middleware is a type of software between an application system and system software, and uses basic services or functions provided by the system software to connect various parts of the application system or different applications on the network, so that the purposes of resource sharing and function sharing can be achieved.
(III) Cross-domain multidimensional big data Engine
The cross-domain multidimensional big data engine is provided with a unified data management platform and a big data engine, and is used for realizing unified management of cross-domain data.
(IV) regional Internet of things sensing system
The regional internet of things sensing system is composed of sensing devices such as pressure sensing devices, humidity sensing devices, image sensing devices, light source sensing devices, infrared sensing devices, temperature sensing devices and the like.
(V) converged service sharing center and converged data innovation center
And the open service gateway is provided with a converged service sharing center and a converged data innovation center. The converged service sharing center can merge the data of each region according to the service classification to create different data sharing centers, for example: personal information center, credit information center, legal information center, financial service center, travel service center, comprehensive treatment service center, space-time service center and other sharing centers, and Internet of things service center.
The fusion data innovation center realizes the innovation application of fusion data through a data fusion system and an AI algorithm system, wherein the AI algorithm system consists of a large-scale visual computing platform, a full-time global traffic perception engine and a progressive video search engine.
The fusion service sharing center and the fusion data creation center can present the processed data through the regional application portal after the data are subjected to fusion processing.
(six) regional application Portal
In the regional application portal, the method mainly comprises the steps of ecological environmental protection, global travel, property cities, enterprise intelligent services, electronic fences, intelligent communities, international talent islands, regional economy and brains, cross-border electronic commerce, cross-domain authentication, electronic fences and the like. When a user enters each plate through the regional application portal, information corresponding to each plate formed by the processed data can be acquired.
Seventh, safe operation and maintenance system
The security operation and maintenance system comprises network and security assurance, multi-cloud management, regional cloud unified management, platform interfaces and the like and is used for guaranteeing the security operation of the whole cloud platform.
Eighth open operation system
The open operation system comprises a unified entrance, an open capability, an operation platform and the like, and is used for establishing the unified entrance of data and accessing the data of each area.
(nine) other Structure
The data are processed through a super computing cluster, an area cloud computing platform and an opentack hardware cluster (an open-source cloud computing management platform project is a combination of a series of software open-source projects).
As shown in fig. 2, the regional internet of things platform 200 is deployed with an internet of things gateway 210 and a cloud server 220, and the internet of things device 201 transmits device data to the cloud server 220 through the internet of things gateway 210.
Referring to fig. 3, a flowchart illustrating steps of a method for data processing according to an embodiment of the present invention may specifically include the following steps:
step 301, the gateway of the internet of things acquires first data acquired by the equipment of the internet of things; the first data are encrypted data of the internet of things equipment;
the internet of things device can acquire first data in real time, when the internet of things device uploads the acquired first data to the internet of things platform in the area, the first data can be encrypted first for data security, and the internet of things gateway can acquire the first data, so that the first data can be processed.
after the first data is obtained, the internet of things gateway can obtain the equipment identifier of the internet of things equipment, and the first data is encrypted by the internet of things equipment, so that the first data can be decrypted by adopting the equipment identifier to obtain the second data.
after the second data is obtained, a large amount of invalid data is contained in the second data, wherein the invalid data can be data with data values not in a preset range, null values or data which do not belong to the area internet of things and are needed.
In the gateway of the internet of things, effective data and attribute information of the effective data can be determined according to user requirements, for example: and the time attribute information and the numerical attribute information can establish a preset filtering rule according to the effective data attribute information, and filter the second data through the preset filtering rule, so as to filter invalid data and obtain third data.
For example, the user needs to determine the attendance information of the employee a to determine whether the employee a is late or early, and the attendance instrument collects the card punching records of the employee a in a plurality of shifts in the day, but in reality, the earliest or latest card punching time can determine whether the employee is late or early. The earliest and latest punch records are valid data, while the other points in time are recorded as invalid data.
Therefore, the preset filtering rule can be set as a rule associated with the time attribute information, the earliest and latest punching records are extracted by time sequencing the plurality of business-on and business-off punching records, and other punching records are deleted, so that the filtering of punching data is realized.
In one example, the preset filtering rules may be established based on numerical properties of the valid data.
For example: the user needs data a, and the data a numerical attributes are: the numerical range is between [10, 20 ].
Assuming that a set of second data is 0, 7, 15, 19, where 0 and 7 are not within [10, 20], 0 and 7 can be determined as invalid data, and the third data obtained after filtering is 15, 19.
in an example, the compression information may include compression format information, compression ratio information, compression speed information.
The correction processing can be to correct the data according to the data format required by the cloud server, so that the data of different Internet of things devices are unified, and the cloud server can process the data conveniently; or deleting the data with logic errors according to the logic relation of the data; duplicate data may also be deduplicated. The correction information is used for representing the correction processing mode which is specifically needed to be carried out on the data. The compression processing can reduce the data transmission quantity, thereby reducing occupied bandwidth and relieving the data pressure of the cloud server.
After the third data is obtained, the gateway of the internet of things can firstly determine the data type of the third data, wherein the third data is effective data subjected to filtering processing, and different correction processing and compression processing can be adopted for data of different data types. Accordingly, after the data type is determined, the correction information and the compression information corresponding to the data type may be determined according to the data type.
after the correction information and the compression information are obtained, the gateway of the internet of things can carry out correction processing on the third data according to the correction information to obtain fourth data. By correcting the data, the effective data is converted into data which can be directly processed by the cloud server, and the data processing pressure of the cloud server is relieved.
And 306, the gateway of the internet of things compresses the fourth data according to the compressed information to obtain target data.
After the fourth data is obtained, the gateway of the internet of things can compress the fourth data according to the compression information to obtain target data. Through compression processing, data volume transmission can be reduced, so that network bandwidth occupied by data transmission is reduced, and the pressure of processing data by a cloud server is relieved.
In an embodiment of the present invention, further includes:
determining encryption information corresponding to the data type; and carrying out encryption processing on the target data according to the encryption information.
After the target data is obtained, in order to ensure the security of the data in the transmission process, encryption information corresponding to the data type of the target data can be determined, and encryption processing is performed on the target data according to the encryption information.
In an embodiment of the present invention, the area internet of things platform is deployed with a cloud server, and further includes:
and sending the target data to the cloud server.
After the target data are obtained, the target data are sent to the cloud server, and the cloud server can obtain the required data after decompressing the target data, so that the data acquired by the Internet of things equipment are transmitted to the cloud server of the regional Internet of things platform after being processed.
According to the embodiment of the invention, the first data acquired by the Internet of things equipment is acquired through the Internet of things gateway; the method comprises the steps that first data are encrypted data of Internet of things equipment, an Internet of things gateway obtains equipment identification of the Internet of things equipment, decrypts the first data according to the equipment identification to obtain second data, the Internet of things gateway filters the second data according to preset filtering rules to obtain third data, the Internet of things gateway determines data types corresponding to the third data and determines correction information and compression information corresponding to the data types, the Internet of things gateway corrects the third data according to the correction information to obtain fourth data, and the Internet of things gateway compresses the fourth data according to the compression information to obtain target data, so that data transmission occupied bandwidth can be reduced, and data processing pressure of a server is reduced.
Referring to fig. 4, a flowchart illustrating steps of another method for processing data according to an embodiment of the present invention may specifically include the following steps:
step 401, the gateway of the internet of things acquires first data acquired by the equipment of the internet of things; the first data are encrypted data of the internet of things equipment;
in an embodiment of the present invention, the data type includes a character type and a non-character type, and the determining, by the gateway of the internet of things, the data type corresponding to the third data and determining the correction information corresponding to the data type includes:
determining a data type corresponding to the third data; judging whether the data type is non-character type; and when the data type is a non-character type, determining that the correction format information corresponding to the data type is a character type.
Wherein, the data type may include a character type and a non-character type; the modification information may be modification format information.
The data type comprises a character type and a non-character type, when the cloud server connected with the gateway of the Internet of things processes the character type, the data type corresponding to the third data can be determined, whether the data type is the non-character type or not is judged, and when the third data is the non-character type, format correction is required to be carried out on the third data, so that correction format information corresponding to the data type can be determined to be the character type.
in an embodiment of the present invention, the correction information includes correction format information, and the performing correction processing on the third data according to the correction information to obtain fourth data includes:
and carrying out format conversion processing on the third data according to the corrected format information to obtain fourth data.
After the correction information and the compression information are determined, carrying out format conversion processing on the third data according to the correction format information to obtain fourth data.
after the fourth data is obtained, the data size of the fourth data can be obtained, and the data size of the fourth data of the filtered and corrected data can be determined.
the preset data amount can be set by a person skilled in the art to a specific value according to the past data transmission amount between the internet of things gateway and the cloud server. After the data amount is acquired, it may be determined whether or not the data amount is greater than a preset data amount.
And step 408, when the data amount is greater than the preset data amount, performing compression processing on the fourth data according to the compression information to obtain target data.
And when the data volume is larger than the preset data volume, compressing the fourth data according to the compression information to obtain target data.
The data with the data quantity larger than the preset data quantity is compressed, so that the data quantity can be reduced, meanwhile, the data with the small data quantity can be directly transmitted, and the compression efficiency of the gateway of the Internet of things is improved.
According to the embodiment of the invention, the first data acquired by the Internet of things equipment is acquired through the Internet of things gateway; the method comprises the steps that first data are encrypted data of Internet of things equipment, an Internet of things gateway obtains equipment identification of the Internet of things equipment, decrypts the first data according to the equipment identification to obtain second data, the Internet of things gateway filters the second data according to preset filtering rules to obtain third data, the Internet of things gateway determines data types corresponding to the third data and determines correction information and compression information corresponding to the data types, the Internet of things gateway corrects the third data according to the correction information to obtain fourth data, and the Internet of things gateway compresses the fourth data according to the compression information to obtain target data, so that data transmission occupied bandwidth can be reduced, and data processing pressure of a server is reduced.
It should be noted that, for simplicity of description, the method embodiments are shown as a series of acts, but it should be understood by those skilled in the art that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.
Referring to fig. 5, a schematic structural diagram of a data processing device according to an embodiment of the present invention is shown, where the device is applied to an area internet of things platform, and the area internet of things platform is deployed with an internet of things gateway, and may specifically include the following modules:
the first data acquisition module 501 is configured to acquire first data acquired by the internet of things device by using the gateway of the internet of things; the first data are encrypted data of the internet of things equipment;
the first data decryption module 502 is configured to obtain, by using the gateway of the internet of things, an equipment identifier of the device of the internet of things, and decrypt the first data according to the equipment identifier to obtain second data;
a second data filtering module 503, configured to filter the second data according to a preset filtering rule by using the internet of things gateway to obtain third data;
a third data processing module 504, configured to determine a data type corresponding to the third data, and determine correction information and compression information corresponding to the data type;
in an example, the compression information may include compression format information, compression ratio information, compression speed information.
In an embodiment of the present invention, the data types include a character type and a non-character type, and the third data processing module 504 may include:
a data type determining submodule, configured to determine a data type corresponding to the third data;
a data type judging sub-module for judging whether the data type is non-character type;
and the correction format information determining submodule is used for determining that the correction format information corresponding to the data type is of a character type when the data type is of a non-character type.
The third data correction module 505 is configured to perform correction processing on the third data by using the internet of things gateway according to the correction information, so as to obtain fourth data;
in an embodiment of the present invention, the correction information includes correction format information, and the third data correction module 505 may include:
and the format conversion sub-module is used for carrying out format conversion processing on the third data according to the corrected format information to obtain fourth data.
And a fourth data compression module 506, configured to compress the fourth data according to the compression information by using the internet of things gateway to obtain target data.
In one embodiment of the present invention, the fourth data compression module 506 may include:
a data volume obtaining sub-module, configured to obtain a data volume of the fourth data;
the data quantity judging sub-module is used for judging whether the data quantity is larger than a preset data quantity or not;
and the compression sub-module is used for compressing the fourth data according to the compression information when the data volume is larger than the preset data volume to obtain target data.
In an embodiment of the present invention, the apparatus may further include:
the encryption information determining module is used for determining encryption information corresponding to the data type;
and the encryption processing module is used for carrying out encryption processing on the target data according to the encryption information.
In an embodiment of the present invention, the area internet of things platform is deployed with a cloud server, and the apparatus may further include:
and the target data sending module is used for sending the target data to the cloud server.
According to the embodiment of the invention, the first data acquired by the Internet of things equipment is acquired through the Internet of things gateway; the method comprises the steps that first data are encrypted data of Internet of things equipment, an Internet of things gateway obtains equipment identification of the Internet of things equipment, decrypts the first data according to the equipment identification to obtain second data, the Internet of things gateway filters the second data according to preset filtering rules to obtain third data, the Internet of things gateway determines data types corresponding to the third data and determines correction information and compression information corresponding to the data types, the Internet of things gateway corrects the third data according to the correction information to obtain fourth data, and the Internet of things gateway compresses the fourth data according to the compression information to obtain target data, so that data transmission occupied bandwidth can be reduced, and data processing pressure of a server is reduced.
An embodiment of the present invention also provides a server, which may include a processor, a memory, and a computer program stored on the memory and capable of running on the processor, the computer program implementing the method of data processing as above when executed by the processor.
An embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements a method of data processing as above.
For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.
The foregoing has outlined rather broadly the more detailed description of a method and apparatus for data processing that uses specific examples to illustrate the principles and embodiments of the present invention, the description of the examples above being provided only to facilitate an understanding of the method and core concepts of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (8)
1. The data processing method is characterized by being applied to an area internet of things platform, wherein an internet of things gateway is deployed on the area internet of things platform, and the method comprises the following steps:
the gateway of the Internet of things acquires first data acquired by equipment of the Internet of things; the first data are encrypted data of the internet of things equipment;
the gateway of the Internet of things acquires the equipment identifier of the equipment of the Internet of things, and decrypts the first data according to the equipment identifier to obtain second data;
the gateway of the Internet of things filters the second data according to preset filtering rules to obtain third data;
the gateway of the Internet of things determines a data type corresponding to the third data and determines correction information and compression information corresponding to the data type; the correction information comprises correction format information;
the gateway of the Internet of things performs format conversion processing on the third data according to the corrected format information to obtain fourth data;
the gateway of the Internet of things compresses the fourth data according to the compressed information to obtain target data;
the data type includes a character type and a non-character type, the gateway of the internet of things determines the data type corresponding to the third data, and determines correction information corresponding to the data type, including:
determining a data type corresponding to the third data;
judging whether the data type is non-character type;
and when the data type is a non-character type, determining that the correction format information corresponding to the data type is a character type.
2. The method of claim 1, wherein the internet of things gateway compresses the fourth data according to the compression information to obtain target data, and the method comprises:
acquiring the data volume of the fourth data;
judging whether the data volume is larger than a preset data volume or not;
and when the data volume is larger than the preset data volume, compressing the fourth data according to the compression information to obtain target data.
3. The method according to claim 1 or 2, further comprising:
determining encryption information corresponding to the data type;
and carrying out encryption processing on the target data according to the encryption information.
4. The method of claim 1, wherein the regional internet of things platform is deployed with a cloud server, further comprising:
and sending the target data to the cloud server.
5. The method of claim 1, wherein the compression information includes compression format information, compression ratio information, compression speed information.
6. The utility model provides a data processing's device, its characterized in that is applied to regional thing networking platform, regional thing networking platform deploys the thing networking gateway, the device includes:
the first data acquisition module is used for acquiring first data acquired by the Internet of things equipment by the Internet of things gateway; the first data are encrypted data of the internet of things equipment;
the first data decryption module is used for acquiring the equipment identifier of the equipment of the Internet of things by the gateway of the Internet of things, decrypting the first data according to the equipment identifier, and obtaining second data;
the second data filtering module is used for filtering the second data according to preset filtering rules by the gateway of the Internet of things to obtain third data;
the third data processing module is used for determining a data type corresponding to the third data and determining correction information and compression information corresponding to the data type;
the third data correction module is used for correcting the third data according to the correction information by the gateway of the Internet of things to obtain fourth data;
the fourth data compression module is used for compressing the fourth data according to the compression information by the Internet of things gateway to obtain target data;
wherein the data type comprises a character type and a non-character type, and the third data processing module comprises:
a data type determining submodule, configured to determine a data type corresponding to the third data;
a data type judging sub-module for judging whether the data type is non-character type;
the correction format information determining submodule is used for determining that the correction format information corresponding to the data type is of a character type when the data type is of a non-character type;
wherein the correction information includes correction format information, and the third data correction module includes:
and the format conversion sub-module is used for carrying out format conversion processing on the third data according to the corrected format information to obtain fourth data.
7. A server comprising a processor, a memory and a computer program stored on the memory and capable of running on the processor, which computer program, when executed by the processor, implements the method of data processing according to any one of claims 1 to 5.
8. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements a method of data processing according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011291328.5A CN112332981B (en) | 2020-11-17 | 2020-11-17 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011291328.5A CN112332981B (en) | 2020-11-17 | 2020-11-17 | Data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112332981A CN112332981A (en) | 2021-02-05 |
| CN112332981B true CN112332981B (en) | 2023-06-06 |
Family
ID=74321217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011291328.5A Active CN112332981B (en) | 2020-11-17 | 2020-11-17 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112332981B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113364617B (en) * | 2021-06-02 | 2022-06-03 | 深圳市优标检测技术有限公司 | Information acquisition method of Internet of things detection equipment |
| CN113411396A (en) * | 2021-06-18 | 2021-09-17 | 泰安易扬科技有限公司 | Intelligent water affair information transmission system, water affair information system and transmission method |
| CN113765934A (en) * | 2021-09-17 | 2021-12-07 | 国网山东省电力公司 | Information sharing and interaction method, device, product and storage medium for distribution network |
| CN114666406B (en) * | 2022-02-24 | 2023-11-21 | 国电南瑞科技股份有限公司 | Electric power Internet of things data compression method and device based on object model |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734024A (en) * | 2018-05-11 | 2018-11-02 | 四川长虹电器股份有限公司 | A kind of efficient database encryption method based on dictionary mapping |
| CN109284619A (en) * | 2018-11-07 | 2019-01-29 | 重庆光电信息研究院有限公司 | The region Locating Type edge calculations system and method for heterologous city Internet of Things |
| CN109862011A (en) * | 2019-02-01 | 2019-06-07 | 华南理工大学 | A kind of environment of internet of things real-time monitoring system calculated based on mist |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10868865B2 (en) * | 2017-11-20 | 2020-12-15 | Moshe Shadmon | System and apparatus to manage data using a peer-to-peer network and the blockchain |
-
2020
- 2020-11-17 CN CN202011291328.5A patent/CN112332981B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734024A (en) * | 2018-05-11 | 2018-11-02 | 四川长虹电器股份有限公司 | A kind of efficient database encryption method based on dictionary mapping |
| CN109284619A (en) * | 2018-11-07 | 2019-01-29 | 重庆光电信息研究院有限公司 | The region Locating Type edge calculations system and method for heterologous city Internet of Things |
| CN109862011A (en) * | 2019-02-01 | 2019-06-07 | 华南理工大学 | A kind of environment of internet of things real-time monitoring system calculated based on mist |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112332981A (en) | 2021-02-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112332981B (en) | Data processing method and device | |
| CN111092938A (en) | Smart city management system based on cloud platform | |
| CN111160869A (en) | Intelligent early warning care system based on community meshing | |
| US11392733B2 (en) | Multi-dimensional event model generation | |
| CN102968494B (en) | The system and method for transport information is gathered by microblogging | |
| CN110913032A (en) | Method and platform for realizing DNS (Domain name System) domain name request analysis by using power grid threat information | |
| US20230394408A1 (en) | Road network operation management method and device, storage medium, and terminal | |
| CN112328585A (en) | Data processing method and device | |
| CN113704567A (en) | Internet of things data management method and device, Internet of things data resource pool and equipment | |
| CN113626648B (en) | Water conservancy data processing system, method and storage medium | |
| CN114841662A (en) | Infrastructure construction project management and control method and device, computer equipment and storage medium | |
| CN114418360A (en) | Smart city operation sign big data analysis method and device | |
| CN115567563B (en) | Comprehensive transportation hub monitoring and early warning system based on end edge cloud and control method thereof | |
| CN112529466A (en) | Smart city management method and system based on big data | |
| CN116028467A (en) | Intelligent service big data modeling method, system, storage medium and computer equipment | |
| CN112383435B (en) | Fault processing method and device | |
| CN112382122B (en) | Traffic information processing method and device | |
| CN112258373A (en) | Data processing method and device | |
| CN112258371A (en) | Fault processing method and device | |
| CN112468696A (en) | Data processing method and device | |
| CN112052338A (en) | Method, system and storage medium for constructing knowledge graph | |
| CN112333199B (en) | Data processing method and device | |
| CN112257674B (en) | Visual data processing method and device | |
| CN112306992B (en) | Big data platform system based on internet | |
| CN116186414B (en) | Entity portrait scheduling system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |