CN113259348A - Heterogeneous data processing method and device, computer equipment and storage medium - Google Patents
Heterogeneous data processing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN113259348A CN113259348A CN202110519051.5A CN202110519051A CN113259348A CN 113259348 A CN113259348 A CN 113259348A CN 202110519051 A CN202110519051 A CN 202110519051A CN 113259348 A CN113259348 A CN 113259348A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- key information
- heterogeneous data
- heterogeneous
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/12—Applying verification of the received information
- H04L63/123—Applying verification of the received information received data contents, e.g. message integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Storage Device Security (AREA)
Abstract
The invention relates to the field of data processing, and discloses a heterogeneous data processing method, a heterogeneous data processing device, a computer device and a storage medium, wherein the method comprises the following steps: by acquiring heterogeneous data, extracting key information of the heterogeneous data based on the data type of the heterogeneous data, encrypting the key information and storing the encrypted key information in a block chain, acquiring the encrypted key information from the block chain to verify the integrity of the heterogeneous data when the integrity of the heterogeneous data needs to be verified, and improving the safety and reliability of the heterogeneous data when a V2X terminal transmits massive heterogeneous data.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a heterogeneous data processing method and apparatus, a computer device, and a storage medium.
Background
At present, a V2X technology (Vehicle to electrical communication technology) can realize high-rate and high-density communication at 250Kph, and support heterogeneous data transmission, and particularly in a commercial application, a V2X terminal as an information interaction key node needs to transmit massive heterogeneous data, which is closely related to road safety, account safety, financial safety, and the like.
In order to ensure the security in the data using process, currently, encryption is often performed when heterogeneous data in vehicle wireless communication is generated, another communication terminal performs decryption for use when receiving encrypted data, along with the development of novel networks such as 5G and the like, the speed of data transmission interaction is substantially improved, so that the data volume to be processed is increased within a certain time, when the data volume is large, the mode of firstly encrypting and then decrypting is adopted, the consumed time is far longer than that of data transmission, and the subsequent data participating in communication cannot be responded in time, so that data processing is abnormal, and therefore a method capable of improving the processing efficiency of the heterogeneous data in the vehicle wireless communication is urgently needed.
Disclosure of Invention
Embodiments of the present invention provide a method and an apparatus for processing heterogeneous data, a computer device, and a storage medium, so as to improve security and reliability of a key when a V2X terminal encrypts and decrypts a large amount of heterogeneous data.
In order to solve the foregoing technical problem, an embodiment of the present application provides a heterogeneous data processing method, including:
obtaining heterogeneous data;
extracting key information of the heterogeneous data based on the data type of the heterogeneous data;
encrypting the key information and storing the key information to a block chain;
and when the integrity of the heterogeneous data is verified, acquiring encrypted key information from the block chain to verify the integrity of the heterogeneous data.
In order to solve the foregoing technical problem, an embodiment of the present application further provides a heterogeneous data processing apparatus, including:
the acquisition module is used for acquiring heterogeneous data;
the extraction module is used for extracting key information of the heterogeneous data based on the data type of the heterogeneous data;
the encryption module is used for encrypting the key information and storing the key information to a block chain;
and the verification module is used for acquiring the encrypted key information from the block chain to verify the integrity of the heterogeneous data when the integrity of the heterogeneous data is verified.
In order to solve the above technical problem, an embodiment of the present application further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the above heterogeneous data processing method when executing the computer program.
In order to solve the above technical problem, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored, and the computer program, when executed by a processor, implements the steps of the above heterogeneous data processing method.
According to the heterogeneous data processing method, the heterogeneous data processing device, the computer equipment and the storage medium, the heterogeneous data are obtained, the key information of the heterogeneous data is extracted based on the data type of the heterogeneous data, the key information is stored in the block chain after being encrypted, when the integrity of the heterogeneous data needs to be verified, the encrypted key information is obtained from the block chain to verify the integrity of the heterogeneous data, and when the V2X terminal transmits massive heterogeneous data, the safety and reliability of the heterogeneous data are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a flow diagram of one embodiment of a heterogeneous data processing method of the present application;
FIG. 3 is a schematic block diagram of one embodiment of a heterogeneous data processing apparatus according to the present application;
FIG. 4 is a schematic block diagram of one embodiment of a computer device according to the present application.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "including" and "having," and any variations thereof, in the description and claims of this application and the description of the above figures are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or in the above-described drawings are used for distinguishing between different objects and not for describing a particular order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, as shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture experts Group Audio Layer III, motion Picture experts compression standard Audio Layer 3), MP4 players (Moving Picture experts Group Audio Layer IV, motion Picture experts compression standard Audio Layer 4), laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
The heterogeneous data processing method provided by the embodiment of the present application is executed by a server, and accordingly, a heterogeneous data processing apparatus is disposed in the server.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. Any number of terminal devices, networks and servers may be provided according to implementation needs, and the terminal devices 101, 102 and 103 in this embodiment may specifically correspond to an application system in actual production.
Referring to fig. 2, fig. 2 shows a heterogeneous data processing method according to an embodiment of the present invention, which is described by taking the method applied to the server in fig. 1 as an example, and is detailed as follows:
s201: and acquiring heterogeneous data.
Specifically, heterogeneous data generated in a commercial application scene of the V2X technology is acquired through a communication network, wherein the application scene comprises but is not limited to formation-holding driving, advanced driving, remote driving, automatic driving, electronic automobile payment, automatic parking, automatic refueling and traffic network planning and control, and the heterogeneous data comprises but is not limited to video, images, voice, text, traffic information and electronic payment data and control instructions.
S202: and extracting key information of the heterogeneous data based on the data type of the heterogeneous data.
Specifically, based on the data type of the heterogeneous data, the information abstraction technology is adopted to extract the information of the heterogeneous data to obtain key information, wherein the data type of the heterogeneous data comprises video data, image data, voice data, text data and other data, wherein the other data comprises but is not limited to traffic information and electronic payment data and control instructions, the key information is data information with a certain length extracted from the heterogeneous data through the information abstraction technology, and the key information is used for ensuring the integrity of the heterogeneous data and preventing the heterogeneous data from being tampered in a commercial application scene in the V2X technology.
S203: and encrypting the key information and storing the key information to the block chain.
S204: and when the integrity of the heterogeneous data needs to be verified, acquiring the encrypted key information from the block chain to verify the integrity of the heterogeneous data.
Specifically, the encrypted key information is obtained from the block chain and decrypted, the decrypted key information is compared with the corresponding data segments in the heterogeneous data, if the data segments are consistent, the heterogeneous data is proved to be complete and reliable, otherwise, the heterogeneous data is proved to be incomplete and possibly tampered.
In the embodiment, by acquiring heterogeneous data, extracting key information of the heterogeneous data based on the data type of the heterogeneous data, encrypting the key information and storing the encrypted key information in the block chain, when the integrity of the heterogeneous data needs to be verified, the encrypted key information is acquired from the block chain to verify the integrity of the heterogeneous data, and when the V2X terminal transmits massive heterogeneous data, the security and reliability of the heterogeneous data are improved.
In some optional implementation manners of this embodiment, in step S202, the data type of the heterogeneous data is picture data, and the step of extracting the key information of the heterogeneous data includes steps S20201 to S20205:
s20201: and acquiring the category information and the hierarchy information of the picture data by adopting a preset judgment mode.
Specifically, the preset determination method may be a similarity calculation method, a clustering algorithm, and the like, wherein the similarity calculation method obtains category information and hierarchy information by calculating similarity values of pixels in the picture data and classifying the pixels in the picture data according to the similarity values, the clustering algorithm obtains the category information and the hierarchy information by clustering the pixels in the picture data, obtains the category information of the picture data by adopting the similarity calculation method, the clustering algorithm, and the like to classify the pixels according to the similarity values, and obtains the hierarchy information by layering the category information. For example, the obtained hierarchy information includes a first hierarchy, a second hierarchy, a third hierarchy, and so on, the category information includes that the number of categories of the first hierarchy is 1, the number of categories of the second hierarchy is 2, the number of categories of the third hierarchy is 3, and so on, and here it should be particularly noted that each hierarchy and each category in each hierarchy may be marked with numbers to implement hierarchy differentiation and differentiation of different categories in different hierarchies.
S20202: and according to the category information and the hierarchy information, clustering the picture data by adopting a clustering algorithm to obtain a clustering center of each category in each hierarchy.
Specifically, the classification sequence number and the hierarchical sequence number of the image data clustering result may be labeled according to the category information and the hierarchical information to obtain a hierarchical sequence number and a category sequence number, and a clustering center C (i, j) of each category of each hierarchy is obtained according to the hierarchical sequence number and the category sequence number, where i is the hierarchical sequence number, j is the category sequence number, i is 1, 2, 3, and so on, and j is 1, 2, 3, and so on.
S20203: and according to the category information and the hierarchy information, sequencing the clustering centers to obtain a clustering center sequence, and generating a clustering center vector according to the clustering center sequence.
Specifically, the classification sequence number and the hierarchical sequence number may be labeled according to the category information and the hierarchical information, so as to obtain the hierarchical sequence number and the category sequence number, and the clustering centers may be sorted according to the hierarchical sequence number and the category sequence number, so as to obtain a clustering center sequence V ═ C (i, j)), which is a clustering center vector.
S20204: and sequentially acquiring the pixel value corresponding to each clustering center according to the sequence of the clustering centers in the clustering center sequence to obtain a pixel value sequence, and generating a pixel value vector according to the pixel value sequence.
Specifically, the pixel value P (i, j) corresponding to each cluster center may be sequentially obtained in the order of the cluster centers in the cluster center sequence, and a pixel value sequence S ═(P (i, j)) may be obtained, where the pixel value sequence S ═(P (i, j)) is a pixel value vector.
S20205: and taking the cluster center vector and the pixel value vector as key information.
In this embodiment, a preset determination manner is adopted to obtain category information and hierarchy information of picture data, a clustering algorithm is adopted to cluster the picture data according to the category information and the hierarchy information to obtain a cluster center of each category in each hierarchy and a pixel value corresponding to the cluster center, the cluster centers are sorted according to the category information and the hierarchy information to obtain a cluster center sequence, a cluster center vector is obtained according to the cluster center sequence, the pixel values are sorted according to the category information and the hierarchy information to obtain a pixel value sequence, a pixel value vector is obtained according to the pixel value sequence, the cluster center vector and the pixel value vector are used as the key information, so that when a V2X terminal encrypts and decrypts the picture data, only the key information needs to be encrypted and decrypted, and occupation of computing resources and storage resources of the V2X terminal is greatly saved, the working efficiency of the V2X terminal is improved.
In some optional implementation manners of this embodiment, in step S202, the data type of the heterogeneous data is video data, and the step of extracting the key information of the heterogeneous data includes steps S2022 to S2024:
s2022: and acquiring vector information of each frame of target image in the video data, wherein the vector information comprises position information, time information and category information of a target object in each frame of target image.
Specifically, position information, time information and category information of a target object in each frame of target image in video data are obtained by adopting a target detection and object classification algorithm, wherein the position information of the target object is a coordinate position of the target object in the target image, the time information is a time point or a time period when the target image appears in the video data, the category information is a category mark of a pixel point of the target object, and vector information is synthesized according to the position information, the time information and the category information of the target object.
Illustratively, the vector information is represented as LT ═ x, y, t, type, where x, y are the position information of the target object, t is the time (also the index of the target image per frame), and type is the category of the target object.
S2023: in a preset period, when the category information of the target object is inconsistent and/or the variation of the position information of the target object exceeds a preset offset and/or the variation of the number of image frames exceeds a preset first threshold, determining the number of image frames which are changed in the preset period and the moment when the number of image frames is changed.
Specifically, assuming that the total duration of the video data is Z, the value range of the preset period Zt is 0< Zt ≦ Z, for example, the total duration Z is 60 minutes, the preset period is 0< Zt ≦ 60, specific values of the preset period may be set according to actual requirements, the preset offset is a threshold for analyzing historical data in the preset period to obtain a variation of position information of the target object, the preset first threshold is a threshold for analyzing historical data in the preset period to obtain a variation of image frame number, and when one or two or three of the variation of position information, the variation of image frame number, and category information changes, the image frame number that changes in the preset period and the time when the image frame number changes are determined.
For example, in the case of detecting the same object in video data, assuming that the amount of change in the position information is D and the preset offset amount is DT, in a preset period, the amount of change D in the position information before and after the T time of the detected object position is | x2-x1| + | y2-y1|, and if D > DT, the number of image frames that change in the preset period and the time at which the number of image frames change are determined;
in the case of detecting the same object in video data, if the type information detected by the detected object before the time T is type1 and the type information detected after the time T is type2, determining the number of image frames changed in a preset period and the time when the number of image frames is changed;
in the case of detecting the same object in video data, assuming that the number of changed image frames is K, and the preset first threshold for the number of changed image frames is KT, if the number of image frames detected by the detected object before the time T is N1, the number of image frames detected after the time T is N2, and the number of changed image frames K ═ N2-N1| > KT, the number of image frames changed within a preset period and the time at which the number of image frames changed are determined.
S2024: and extracting information according to the number of the image frames and the time when the number of the image frames changes to obtain key information.
Specifically, assuming that the number of changed image frames is K and the time when the number of changed image frames is C, the number of image frames between t (C-K/2) and t (C + K/2) is extracted to obtain the key information.
In this embodiment, the key information is obtained by extracting the information from the video data, so that when the V2X terminal encrypts and decrypts the video data, only the key information needs to be encrypted and decrypted, thereby greatly saving the occupation of the computing resources and the storage resources of the V2X terminal, and improving the working efficiency of the V2X terminal.
In some optional implementation manners of this embodiment, in step S202, the data type of the heterogeneous data is voice data, and the step of extracting the key information of the heterogeneous data includes S2025 to S2026:
s2025: and performing difference calculation on the voice data of the current time period and the voice data of the previous time period to obtain data difference information.
Specifically, for the same segment of voice data, data indexes such as center frequency, wave crest, wave trough, amplitude and the like of the voice data of the current time segment and the voice data of the previous time segment are detected, and difference calculation is performed on the data indexes of the voice data of the current time segment and the voice data of the previous time segment to obtain data difference information.
S2026: and if the data difference information exceeds a preset second threshold, extracting information from the voice data to obtain key information.
Specifically, the preset second threshold is obtained by analyzing the historical data, and if the data difference information exceeds the preset second threshold, the key information abstraction technology is adopted to extract the information of the voice data to obtain the key information.
In this embodiment, the key information is obtained by extracting the information from the voice data, so that when the V2X terminal encrypts and decrypts the voice data, only the key information needs to be encrypted and decrypted, occupation of computing resources and storage resources of the V2X terminal is greatly saved, and working efficiency of the V2X terminal is improved.
In some optional implementation manners of this embodiment, in step S202, the data type of the heterogeneous data is text data, and the step of extracting the key information of the heterogeneous data includes S2027 to S2030:
s2027: and carrying out word segmentation frequency statistics on the text data to obtain a word segmentation frequency statistical result.
Specifically, word frequency statistics is carried out on the text data by adopting a data analysis processing algorithm to obtain word segmentation frequency statistical results.
S2028: and inputting the text data into a preset text classification model to obtain a text classification result.
Specifically, the preset text classification model is obtained by pre-training the historical text data, and words, phrases, sentences, semantics and grammatical structures in the text data can be analyzed, so that the text data can be input into the preset text classification model, and a text classification result is obtained.
S2029: and extracting the character string with the preset length from the text data to obtain the character string with the preset length.
Specifically, a key information summarization technology may be adopted to extract a character string of a preset length from the text data.
S2030: and taking the word segmentation frequency statistical result, the text classification result and the character string with the preset length as key information.
In this embodiment, the key information is obtained by extracting the information from the text data, so that the V2X terminal only needs to encrypt and decrypt the key information when encrypting and decrypting the text data, thereby greatly saving the occupation of the computing resources and the storage resources of the V2X terminal, and improving the working efficiency of the V2X terminal.
Further, when the type of the heterogeneous data is other data such as relational data, non-relational data and the like, the information can be directly extracted by using a SM3 cryptographic algorithm to obtain key information, wherein the SM3 cryptographic algorithm is a cryptographic hash algorithm, is suitable for generation and verification of a digital signature and a verification message authentication code in commercial cryptographic application and generation of a random number, and is used for performing message digest implementation information extraction on other data such as relational data, non-relational data and the like to obtain the key information.
In some optional implementation manners of this embodiment, in step S203, the step of encrypting the key information and storing the encrypted key information in the blockchain includes:
s2031: and based on the national cryptographic algorithm, carrying out asymmetric encryption on the key information to obtain encrypted key information.
Specifically, the key information is asymmetrically encrypted by using the SM2 algorithm, so that the safety of the key information is ensured, and the data volume of the key information is far smaller than that of original heterogeneous data, so that the time consumption required by encryption is reduced when the key information is asymmetrically encrypted, and the calculation efficiency of the V2X terminal is improved, wherein the SM2 algorithm is an elliptic curve public key cryptographic algorithm and is a more advanced and safe algorithm, and is used for replacing an RSA algorithm in a commercial cryptographic system of China.
Optionally, before the key information is asymmetrically encrypted by using the national secret SM2 algorithm, the key information may be symmetrically encrypted by using the national secret SM1 hardware, and the key information is symmetrically encrypted by using a hardware method, where the national secret SM1 algorithm is a commercial cipher block standard symmetric algorithm, and has a characteristic that the algorithm is not public, and only the hardware can be used to realize the encryption function, so as to further improve the security of the key information.
S2032: storing the encryption key information to the blockchain.
Specifically, in step S2032, the encryption key information is stored in the block chain, and since the data amount of the key information is far smaller than the data amount of the original heterogeneous data, the generated block is far smaller than the block required for directly encrypting the original heterogeneous data and storing the original heterogeneous data, a large amount of storage space is saved, and the integrity of the original heterogeneous data can be verified by the encryption key information stored in the block chain, so as to ensure the security of the original heterogeneous data.
Optionally, the original heterogeneous data may be stored in a non-encrypted form in a corresponding consensus block on the block chain, so that the storage space of the block chain is saved, and the integrity of the original heterogeneous data is verified by obtaining the key information corresponding to the original heterogeneous data on the block chain, so as to ensure the security of the original heterogeneous data.
In the embodiment, the key information is encrypted by adopting a national encryption algorithm and then stored in the block chain, so that the security of the key information is improved, and the key information is guaranteed not to be tampered, real and effective.
In some optional implementation manners of this embodiment, after the step of extracting key information of the heterogeneous data based on the data type of the heterogeneous data, the heterogeneous data processing method further includes:
calculating the data volume of the key information;
and if the data volume exceeds the preset third threshold, reducing the data volume until the data volume is lower than the preset third threshold.
Specifically, the preset third threshold is obtained by analyzing the historical data, and if the data volume of the key information exceeds the preset third threshold, the key information can be compressed by adopting a compression algorithm until the data volume of the key information is lower than the preset third threshold; or presetting a random number range, randomly selecting a random number in the random number range, taking the random number as the data length of the data segment, and extracting the data segment in the key information until the data volume of the key information is lower than a preset third threshold value.
In this embodiment, by calculating the data volume of the key information, if the data volume of the key information exceeds the preset third threshold, the data volume of the key information is reduced until the data volume is lower than the preset third threshold, the data volume of the key information can be controlled, the working efficiency of the V2X terminal is improved, and the key information is guaranteed to be true and effective.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
Fig. 3 is a schematic block diagram of a heterogeneous data processing apparatus corresponding to the above-described heterogeneous data processing methods. As shown in fig. 3, the heterogeneous data processing apparatus includes an acquisition module 31, an extraction module 32, an encryption module 33, and a verification module 34. The functional modules are explained in detail as follows:
an obtaining module 31, configured to obtain heterogeneous data.
And the extraction module 32 is configured to extract key information of the heterogeneous data based on the data type of the heterogeneous data.
And the encryption module 33 is configured to encrypt the key information and store the encrypted key information in the block chain.
And the verification module 34 is configured to obtain the encrypted key information from the blockchain to verify the integrity of the heterogeneous data when verifying the integrity of the heterogeneous data.
In one embodiment, the extraction module 32 includes a first extraction unit, which is described in detail below:
and the acquisition unit is used for acquiring the category information and the hierarchy information of the picture data by adopting a preset judgment mode.
And the clustering unit is used for clustering the picture data by adopting a clustering algorithm according to the category information and the hierarchy information to obtain a clustering center of each category in each hierarchy.
And the first sequencing unit is used for sequencing the clustering centers according to the category information and the hierarchy information to obtain a clustering center sequence, and generating a clustering center vector according to the clustering center sequence.
And the second sequencing unit is used for sequentially acquiring the pixel value corresponding to each clustering center according to the sequence of the clustering centers in the clustering center sequence to obtain a pixel value sequence and generating a pixel value vector according to the pixel value sequence.
And the first key information confirming unit is used for taking the clustering center vector and the pixel value vector as key information.
In one embodiment, the extracting module 32 includes a vector information obtaining unit, a determining unit and a second extracting unit, and each unit is described in detail as follows:
the vector information acquisition unit is used for acquiring vector information of each frame of target image in the video data, and the vector information comprises position information, time information and category information of a target object in each frame of target image.
And the determining unit is used for determining the number of image frames changed in the preset period and the moment when the number of image frames is changed when the category information of the target object is inconsistent and/or the variation of the position information of the target object exceeds a preset offset and/or the variation of the number of image frames exceeds a preset first threshold in the preset period.
And the second extraction unit is used for extracting information according to the number of the image frames and the time when the number of the image frames changes to obtain the key information.
In one embodiment, the extraction module 32 includes a difference calculation unit and a third extraction unit, each of which is described in detail as follows:
and the difference value calculating unit is used for calculating the difference value between the voice data of the current time period and the voice data of the previous time period to obtain data difference value information.
And the third extraction unit is used for extracting information from the voice data to obtain key information if the data difference information exceeds a preset second threshold.
In one embodiment, the extracting module 32 includes a frequency statistics unit, a text classification unit, a fourth extracting unit, and a fifth extracting unit, each of which is described in detail as follows:
and the frequency counting unit is used for carrying out word segmentation frequency counting on the text data to obtain a word segmentation frequency counting result.
And the text classification unit is used for inputting the text data into a preset text classification model to obtain a text classification result.
The fourth extraction unit is used for extracting the character strings with preset lengths from the text data to obtain the character strings with the preset lengths;
and the second key information confirming unit is used for taking the word segmentation frequency statistical result, the text classification result and the character string with the preset length as key information.
In one embodiment, the encryption module 33 includes asymmetric encryption and storage units, each of which is described in detail below:
and the asymmetric encryption unit is used for performing asymmetric encryption on the key information based on a national cryptographic algorithm to obtain encrypted key information.
And the storage unit is used for storing the encryption key information to the block chain.
In an embodiment, the heterogeneous data processing apparatus further includes a data amount calculation unit and a data amount reduction unit, and the detailed description of each unit is as follows:
and a data amount calculation unit for calculating a data amount of the key information.
And the data volume reducing unit is used for reducing the data volume if the data volume exceeds a preset third threshold value until the data volume is lower than the preset third threshold value.
For specific limitations of the heterogeneous data processing apparatus, reference may be made to the above limitations of the heterogeneous data processing method, which is not described herein again. The various modules in the heterogeneous data processing apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In order to solve the technical problem, an embodiment of the present application further provides a computer device. Referring to fig. 4, fig. 4 is a block diagram of a basic structure of a computer device according to the present embodiment.
The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It is noted that only the computer device 4 having the components connection memory 41, processor 42, network interface 43 is shown, but it is understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead. As will be understood by those skilled in the art, the computer device is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The computer device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The computer equipment can carry out man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch panel or voice control equipment and the like.
The memory 41 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or D interface display memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the computer device 4. Of course, the memory 41 may also include both internal and external storage devices of the computer device 4. In this embodiment, the memory 41 is generally used for storing an operating system installed in the computer device 4 and various types of application software, such as program codes for controlling electronic files. Further, the memory 41 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 42 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute the program code stored in the memory 41 or process data, such as program code for executing control of an electronic file.
The network interface 43 may comprise a wireless network interface or a wired network interface, and the network interface 43 is generally used for establishing communication connection between the computer device 4 and other electronic devices.
The present application further provides another embodiment, which is to provide a computer-readable storage medium storing an interface display program, which is executable by at least one processor to cause the at least one processor to perform the steps of the heterogeneous data processing method as described above.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.
It is to be understood that the above-described embodiments are merely illustrative of some, but not restrictive, of the broad invention, and that the appended drawings illustrate preferred embodiments of the invention and do not limit the scope of the invention. This application is capable of embodiments in many different forms and is provided for the purpose of enabling a thorough understanding of the disclosure of the application. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to one skilled in the art that the present application may be practiced without modification or with equivalents of some of the features described in the foregoing embodiments. All equivalent structures made by using the contents of the specification and the drawings of the present application are directly or indirectly applied to other related technical fields and are within the protection scope of the present application.
Claims (10)
1. A method for processing heterogeneous data, the method comprising:
obtaining heterogeneous data;
extracting key information of the heterogeneous data based on the data type of the heterogeneous data;
encrypting the key information and storing the key information to a block chain;
and when the integrity of the heterogeneous data is verified, acquiring encrypted key information from the block chain to verify the integrity of the heterogeneous data.
2. The heterogeneous data processing method according to claim 1, wherein the data type of the heterogeneous data is picture data, and the step of extracting key information of the heterogeneous data comprises:
acquiring category information and hierarchy information of the picture data by adopting a preset judgment mode;
clustering the picture data by adopting a clustering algorithm according to the category information and the hierarchy information to obtain a clustering center of each category in each hierarchy;
according to the category information and the hierarchy information, sorting the clustering centers to obtain a clustering center sequence, and generating a clustering center vector according to the clustering center sequence;
sequentially acquiring a pixel value corresponding to each clustering center according to the sequence of the clustering centers in the clustering center sequence to obtain a pixel value sequence, and generating a pixel value vector according to the pixel value sequence;
and taking the clustering center vector and the pixel value vector as the key information.
3. The heterogeneous data processing method according to claim 1, wherein the data type of the heterogeneous data is video data, and the step of extracting the key information of the heterogeneous data comprises:
acquiring vector information of each frame of target image in the video data, wherein the vector information comprises position information, time information and category information of a target object in each frame of target image;
in a preset period, when the category information of the target object is inconsistent and/or the variation of the position information of the target object exceeds a preset offset and/or the variation of the image frame number exceeds a preset first threshold, determining the image frame number which is changed in the preset period and the moment when the image frame number is changed;
and extracting information according to the image frame number and the moment when the image frame number changes to obtain the key information.
4. The heterogeneous data processing method according to claim 1, wherein the data type of the heterogeneous data is voice data, and the step of extracting the key information of the heterogeneous data comprises:
performing difference calculation on the voice data of the current time period and the voice data of the previous time period to obtain data difference information;
and if the data difference information exceeds a preset second threshold, extracting information from the voice data to obtain the key information.
5. The heterogeneous data processing method according to claim 1, wherein the data type of the heterogeneous data is text data, and the step of extracting key information of the heterogeneous data includes:
performing word segmentation frequency statistics on the text data to obtain a word segmentation frequency statistical result;
inputting the text data into a preset text classification model to obtain a text classification result;
extracting a character string with a preset length from the text data to obtain the character string with the preset length;
and taking the word segmentation frequency statistical result, the text classification result and the character string with the preset length as the key information.
6. The heterogeneous data processing method according to claim 1, wherein the step of encrypting the key information and storing the key information in a blockchain comprises:
based on a national cryptographic algorithm, carrying out asymmetric encryption on the key information to obtain encrypted key information;
storing the encryption key information to the blockchain.
7. The heterogeneous data processing method of claim 1, wherein after the step of extracting key information of the heterogeneous data based on the data type of the heterogeneous data, the method further comprises:
calculating the data volume of the key information;
and if the data volume exceeds a preset third threshold, reducing the data volume until the data volume is lower than the preset third threshold.
8. The heterogeneous data processing method device is characterized by comprising the following steps:
the acquisition module is used for acquiring heterogeneous data;
the extraction module is used for extracting key information of the heterogeneous data based on the data type of the heterogeneous data;
the encryption module is used for encrypting the key information and storing the key information to a block chain;
and the verification module is used for acquiring the encrypted key information from the block chain to verify the integrity of the heterogeneous data when the integrity of the heterogeneous data needs to be verified.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the heterogeneous data processing method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the heterogeneous data processing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110519051.5A CN113259348A (en) | 2021-05-12 | 2021-05-12 | Heterogeneous data processing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110519051.5A CN113259348A (en) | 2021-05-12 | 2021-05-12 | Heterogeneous data processing method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113259348A true CN113259348A (en) | 2021-08-13 |
Family
ID=77223246
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110519051.5A Pending CN113259348A (en) | 2021-05-12 | 2021-05-12 | Heterogeneous data processing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113259348A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114168218A (en) * | 2021-11-30 | 2022-03-11 | 深圳市名竹科技有限公司 | Data processing method, data processing device, computer equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102893535A (en) * | 2010-03-12 | 2013-01-23 | 韩国电子通信研究院 | Method and apparatus for transmitting and receiving data in a mimo system |
| KR102010571B1 (en) * | 2017-11-15 | 2019-08-13 | 한전케이디엔 주식회사 | Method for storing smart grid heterogeneous data security using block chain |
| CN112130999A (en) * | 2020-09-23 | 2020-12-25 | 南方电网科学研究院有限责任公司 | Electric power heterogeneous data processing method based on edge calculation |
| CN112732835A (en) * | 2021-01-12 | 2021-04-30 | 宁波云麟信息科技有限公司 | Block chain-based heterogeneous data storage method and electronic equipment |
-
2021
- 2021-05-12 CN CN202110519051.5A patent/CN113259348A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102893535A (en) * | 2010-03-12 | 2013-01-23 | 韩国电子通信研究院 | Method and apparatus for transmitting and receiving data in a mimo system |
| KR102010571B1 (en) * | 2017-11-15 | 2019-08-13 | 한전케이디엔 주식회사 | Method for storing smart grid heterogeneous data security using block chain |
| CN112130999A (en) * | 2020-09-23 | 2020-12-25 | 南方电网科学研究院有限责任公司 | Electric power heterogeneous data processing method based on edge calculation |
| CN112732835A (en) * | 2021-01-12 | 2021-04-30 | 宁波云麟信息科技有限公司 | Block chain-based heterogeneous data storage method and electronic equipment |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114168218A (en) * | 2021-11-30 | 2022-03-11 | 深圳市名竹科技有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN114168218B (en) * | 2021-11-30 | 2024-03-29 | 深圳市名竹科技有限公司 | Data processing method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111080295B (en) | Electronic contract processing method and device based on blockchain | |
| CN112508075B (en) | DBSCAN clustering method based on transverse federation and related equipment thereof | |
| CN108848058A (en) | Intelligent contract processing method and block catenary system | |
| CN113761577B (en) | Big data desensitization method, device, computer equipment and storage medium | |
| CN110532165B (en) | Application program installation package characteristic detection method, device, equipment and storage medium | |
| CN114826553A (en) | Cloud storage data security protection method and device based on group signature and homomorphic encryption | |
| CN113704781A (en) | File secure transmission method and device, electronic equipment and computer storage medium | |
| CN103136459A (en) | Copyright identification method and system for encrypted digital contents | |
| CN116090024B (en) | Reliable data storage device, system and method | |
| CN104346550A (en) | Information processing method and electronic equipment | |
| CN115659378A (en) | Case record information evidence storing method and related equipment | |
| CN114626079A (en) | File viewing method, device, equipment and storage medium based on user permission | |
| CN114666135A (en) | Data encryption method and device, electronic equipment and storage medium | |
| CN113259348A (en) | Heterogeneous data processing method and device, computer equipment and storage medium | |
| CN110069907A (en) | Big data source tracing method and system based on digital watermarking | |
| CN116668188A (en) | Financial data hierarchical encryption method and system | |
| CN113434824B (en) | Software service authorization management method, device, equipment and storage medium | |
| CN113537982B (en) | Security verification method, device, equipment and storage medium of financial equipment | |
| CN115205089A (en) | Image encryption method, network model training method and device and electronic equipment | |
| KR102133902B1 (en) | Secure input device | |
| CN114077749A (en) | Data processing method based on block chain and related equipment thereof | |
| CN106411888A (en) | Sign-off document encryption device and method | |
| CN104732137A (en) | Android-based method of recognizing multi-mode AES-encrypted (advanced encryption standard encrypted) QR (quick response) codes | |
| CN110334815A (en) | Label complementing method, terminal, device and storage medium based on cross validation | |
| CN117972757B (en) | Method and system for realizing safety analysis of mine data based on cloud platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210813 |
|
| RJ01 | Rejection of invention patent application after publication |